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FOREWORD 


One  of  the  outstanding  events  at  the  Sixteenth  Conference  on  the 
Design  of  Experiments  In  Army  Research,  Development  and  Testing  was  the 
announcement  that  Professor  George  Snedecor  was  selected  to  receive  the 
1970  Wilks  Award.  The  remarks  made  by  Professor  0.  Kemp thorns,  who 
accepted  this  award  on  behalf  of  Professor  Snedecor,  are  recorded  In  these 
Proceedings.  Other  Important  evanta  were  the  addresses  made  by  Che 
Invited  speakers.  The  general  areas  covered  by  these  gentlemen  can  be 
gleaned  from  the  titles  of  their  addrassesi 

Minimum  Discrimination  Information  Estimation  and  Application 
Professor  Solomon  Kullhack,  Georgs  Washington  University 

Field  Testing 

Dr.  Richard  J.  Kaplan,  The  RAND  Corporation 

The  Analysis  of  Complex  Contingency  Table  Data  from  General 

Experimental  Designs  and  Sample  Surveys 

Professor  Gary  G.  Koch,  University  of  North  Carols..-' 

Nonparametr: c  Analysis  of  Covariance 

Professor  Dana  Quads,  University  of  North  Carolina 

There  la  little  doubt  that  the  most  valuable  phases  of  these  con¬ 
ferences  are  the  technical  and  clinical  sessions.  Army  scientists  take 
advantage  o  the  technical  sessions  to  announce  their  successes  In  con¬ 
ducting  various  types  of  expsriments,  and  sharing  their  findings  with 
persons  In  other  Installations.  In  the  clinical  sessions  stimulating 
discussions  take  place,  and  these  arguments  often  lead  to  suggestions  on 
how  better  to  analyse  the  troublesome  design  problems  that  are  being 
considered,  This  year  there  were  twenty-four  (24)  papers  for  the  technical 
sessions  and  five  (5)  clinical  papers.  All  these  papers  were  well 
received  and  moat  of  them  appear  as  articles  in  these  Proceedings. 

Colonel  W.  L.  Tate,  Commandant  of  the  U.  S.  Army  Logistics  Management 
Center,  offered  to  hold  the  Sixteenth  Conference  on  the  Design  of  Experi¬ 
ments  in  the  excellent  facilities  at  his  Installation.  He  named  Mrs. 
Virginia  W.  Perry  to  serve  as  the  Chairman  on  Local  Arrangements.  She 
and  members  of  her  committee— Messrs .  C.  A.  Correia,  J.  W.  Griswold, 

E.  <v.  Hartley,  and  R.  L.  Launer — are  to  be  congratulated  on  the  way  they 
harried  the  many  problems  that  arose  before  and  during  the  course  of  the 
symposium. 
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On  bahalf  of  tho  Army  Mat  Kama  tic*  Steering  Committee,  aponaor  of  these 
conference*,  let  at  thank  tha  aany  speekera,  chairmen  and  pant lit ta  for 
all  tha  time  and  effort  they  contributed  to  this  affair.  Without  their 
help  and  the  many  argument a  and  cowenta  aupplied  by  thoae  in  attendance, 
thla  Beating  would  not  have  aerved  lta  main  purpoaee.  At  thia  tine  let  ae 
alao  atata  that  such  credit  for  the  auccaaa  of  thla  conference  la  due  to 
the  aanbera  of  ay  Program  Coaalttae  (Robert  Bechhofer,  Prancla  Dreaael, 
Walter  D.  Foster,  Fred  Priahman,  Hurray  Gelaler,  Boyd  Harahbarger, 

Clifford  Maloney,  Henry  B.  Mann,  William  H,  Marlow,  George  Micholaon, 
Virginia  W.  Perry,  and  Herbert  Solomon),  Finally,  t  would  like  to  thank 
Francis  Dreaael  in  particular  for  the  smooth  accomplishment  of  another 
monumental  job  again  for  this  conference  year. 


Frank  E.  Grubbs 
Conference  Chairman 
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Minimum  Discrimination  Information  Estimation  and  Application 


S.  Kulloactc 

Tne  George  Vfasnington  University 
Wasnington,  D.  C.  20006 

Abstract 

Tnis  paper  presents  in  some  detail  tne  application  of 
tne  minimum  discrimination  information  tneorem  to  tne  analysis 
of  multidimensional  contingency  tades.  It  is  snown  tnat  tne 
form  of  tne  minimum  discrimination  information  estimate  as  a 
memo er  of  an  exponential  family  provides  a  regression 
expression  for  tne  logaritnm  of  tne  estimate.  Computational 
procedures  for  tne  evaluation  of  tne  regression  parameters  and 
tne  minimum  discrimination  information  estimates  are  described 
along  witn  tne  tests  for  tne  nypotneses  as  provided  oy  tne 
minimum  discrimination  information  statistics. 

0.  Introduction.  Tnis  paper  is  related  to  [91  and  [101  in  wnicn 
certain  basic  tecnniques  and  procedures  were  presented  for  tne 

1  Supported  in  part  Dy  tne  Air  Force  Office  of  Scientific  Researcn, 
Office  of  Aerospace  Researcn,  United  States  Air  Force,  under 
Grant  AFOSR-68-1513. 

Thi.«  paper  was  reproduced  photographically  from  the  author’s  manuscript. 
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analysis  of  multidimensional  contingency  tables.  In  tnis  paper 
we  snail  examine  tne  underlying  tneory  in  greater  detail  and 
present  one  Important  area  of  application.  In  particular  we 
snail  detail  tne  close  analogy  of  tnis  application  wltn 
multivariate  regression  analysis.  Altnougn  tne  ingredients  of 
tne  underlying  tneory  were  discussed  In  till  It  seems  necessary 
and  desirable  to  present  tnese  ideas  nere  m  greater  detail. 

We  also  remartc  tnat  a  more  extensive  computer  program  tnan 
tnat  described  in  [6]  and  £91  nas  been  prepared  by  Professor 
Ireland  of  Tne  George  Wasnington  University.  Tnis  new  program 
can  nandle  tables  of  nigner  dimension  tnan  four- way  contingency 
taoles  and  also  provides  tne  values  of  additional  useful 
parameters. 

It  snould  oe  pointed  out  tnat  tnere  are  otner  areas  of 
application  of  minimum  discrimination  information  estimation 
tnan  tnat  considered  in  dotail  in  tnis  paper,  for  example,  [3], 
[4],  [51,  [71,  [11],  [12],  [13],  [1^] .  Tne  particular 
application  we  snail  consider  nere  can  be  described  as  fitting 
tne  observed  values  in  tne  cells  of  a  contingency  table  in 
terms  of  a  regression  based  on  sets  of  observed  marginals  as 
explanatory  variables. 

1.  Discrimination  information.  To  maxe  tne  discussion  more 
specific  we  snail  present  it  in  terms  of  tne  analysis  of  four¬ 
way  contingency  tables.  All  tne  essential  features  of  a  more 
general  presentation  appear.  Let  us  consider  tne  space  n 
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of  four-way  contingency  tables  R  x  S  *  T  x  U  of  dimension 
rxSxtxuao  tnat  tne  generic  variaole  la  tv  •  (i,j,K,t), 
i  *  *  *  *  i  0  *  ^i***!®!  xc  •  1|  1 1  •  |t|  t  *  1|  •  •  *  |U«  Suppose 

tnere  are  two  probability  distributions  or  contingency  tables 
(  we  snail  use  tnese  terms  interchangeably  )  defined  over  tne 
space  n,  eay  p(w),  tt(w),  ^  p(»)  -  1,  ^  n(»)  -  1.  Tne 
discrimination  information  is  defined  by 


(1.1)  I(PSTT)  m  X  P(tu)  ■tn  ^  W  • 

f)  TT  (tli ) 


Tne  basis  for  tnis  definition,  its  properties,  and  relation  to 
otner  definitions  of  information  measures  may  be  found  in  [11], 
in  tne  Proceedings  of  [131  and  references  tnerein.  For  tne 
particular  types  of  application  of  Interest  nere  tne  ir-distributlon, 
n(w),  in  tne  definition  (1.1)  according  to  tne  proDlem  of 
interest  may  eitner  be  specified,  or  it  may  be  an  estimated 
distribution,  or  it  may  be  an  observed  distribution.  Tne 
p-distrlbution,  p(ui),  in  tne  definition  (1.1)  ranges  over  or 
is  a  member  of  a  family  of  distributions  of  interest. 

Of  tne  various  properties  of  I(p:tt)  we  mention  in  particular 
tne  fact  tnat  I(p:n)  >  0  and  -  0  if  and  only  if  p(u>)  -  rr(iu) . 


2.  Minimum  discrimination  information  estimation.  Many 
problems  in  tne  analysis  of  contingency  tables  may  be  characterized 
as  estlrving  a  distribution  or  contingency  table  subject  to 
certain  restraints  and  tnen  comparing  tne  estimated  table  wltn  an 
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observed  table  to  determine  wnetner  tne  observed  table 
satisfies  a  null  nypotneais  implied  by  tne  restraints.  In 
accordance  wltn  tne  principle  of  minimum  discrimination 
Information  estimation  we  select  tnat  member  of  tne  family 
of  p-dlatrlbutlons  satisfying  tne  restraints  wnicn  minimises  - 
tne  discrimination  Information  I(pm)  over  tne  f  stilly  of 
pertinent  p-distributions.  We  denote  tne  minimum  discrimination 
information  estimate  by  p*(u>)  so  tnat 

(2.1)  I(p*wr)  -  I  m  -  min  I  (pm). 

TT  (til) 

Unless  otherwise  stated,  tne  summation  Is  over  0  wnicn  will  be 
omitted. 

In  one  class  of  proolems  tne  restraints  specify  some 
requirement  external  to  tne  observed  values,  for  example,  tnat 
a  set  of  marginals, nave  specified  values  as  determined  by  genetic 
or  otner  tneory  [4],  C5l,  [12],  or  tnat  marginals  be  nomogeneous 
[31,  [14],  or  tnat  tne  distribution  satisfy  certain  symmetry 
conditions  [3].  In  sucn  problems  tt(»)  is  taxen  to  De  an  observed 
contingency  table,  tnat  Is,  x(u>)  -  x(ijxt)  -  m(ijxt),  wnere 
n  -  2  x(u>) . 

In  anotner  class  of  problems  tne  restraints  specify  tnat  tne 
estimated  distribution  or  contingency  table  nave  some  set  of 
marginals  wnicn  are  tne  same  as  tnose  of  an  observed  contingency 
taDle.  In  sucn  cases  tt  (uu)  is  taxen  to  be  eitner  tne  uniform 
distribution  Tr(ijxt)  -  l/rstu  or  &  distribution  already  estimated 
subject  to  restraints  contained  in  and  implied  Dy  tne  restraints 
under  examination.  Tne  latter  case  includes  tne  classical 
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nypotneses  of  independence,  conditional  independence,  Homogeneity, 
conditional  nomogenelty  and  Interaction,  all  of  wnicn  can  be 
considered  as  instances  of  generalised  Independence  [3],  [61, 

C7],  C61,  [9I,  ClOl,  [131,  and  *111  be  considered  in  some  detail 
in  tnis  paper, 

3.  Minimum  discrimination  information  statistic*  To  test 
wnetner  an  ooserved  contingency  table  satisfies  tne  null 
nypotnesis  as  represented  by  tne  minimum  discrimination 
information  estimate  we  compute  a  measure  of  tne  deviation 
Dctween  tne  observed  distribution  and  tne  appropriate  estimate 
oy  tne  minimum  discrimination  information  statistic.  For 
notatlonal  convenience  and  later  computational  convenience  let 
us  denote  tne  estimated  contingency  t&ole  in  terms  or 
occurrences  Dy  **(uu)  -  np*(uj),  tnen  for  tne  first  category  of 
problems,  tnat  is,  witn  restraints  determined  oy  external 
considerations,  tne  minimum  discrimination  information  statistic 
turns  out  to  be 

(3.1)  2I(x*;x)  -  2Z  x*(u-)  in  x 

x(w) 

wnicn  is  asymptotically  distributed  as  a  X*  witn  appropriate 
degrees  of  freedom  under  tne  null  nypotnesis.  For  tne  second 
category  of  problems,  tnat  is,  witn  tne  restraints  implied  oy 
e  set  of  observed  marginals,  or  tnose  of  a  generalized 
Independence  nypotnesis,  tne  m.d.l.  statistic  is 


' ?  "fTTfT  j'!pwif«iif,:*»^r#|i 1  jjjjh  |||f  i  !|  .  TW* ^*n 


(3.2)  21{xtx*)  -  2Z  x(«)  in 


wnicn  is  asymptotically  distributed  as  a  x*  witn  appropriate 
; agrees  of  freedom  under  tne  null  nypotnesia. 

Tne  statistic  in  (3.2)  is  also  minus  twice  tne  logarltna 
of  tne  llKSllnood  ratio  statistic  but  tnls  is  not  true  for 
tne  statistic  in  (3>1)  or  in  otner  applications  [11]. 


4.  Minimum  discrimination  information  tneorcn.  We  now  present 
i x  tneorem  wnicn  is  tne  oasis  for  tne  principle  of  minimum 
discrimination  information  estimation  and  its  applications.  Ve 
snail  present  it  in  a  form  related  to  tne  context  of  tnis 
discussion  on  tne  analysis  of  contingency  taDles. 

Let  us  consider  tne  space  n  mentioned  in  section  1  and 
tne  discrimination  information  introduced  in  (1.1).  Suppose 
now,  for  example,  tnat  we  nave  tnree  linearly  independent 
statistics  of  interest  defined  over  tne  space  n 


(4.1)  Tj  (uj)  ,  T#  (ui) ,  T#  (u)) 


Let  us  determine  tne  value  of  p(te)  wnicn  minimises  tr.e 
discrimination  information 


(4.2)  I (p :tt)  -  Z  p (uu)  < ,n  £ijsl 

tt(uj) 


over  tne  family  of  p-distributlens  wnicn  satisfy  tne  restraints 


i 


<*,i r-*!-  . . -t  .-r  ....  .....  . . ..v^mA'riwwMiritfini  r/.sueeurnr'iflf*. 


Z  T,  {«)  P(»)  -  9* 

(*.3)  z  i; («)  »(«)  -  9J 

z  Tg(®)  p(a»)  -  9J 

wnere  €f,  9f  are  specified  values. 

If  n(«)  satisfies  tne  restraints  v  *3)  tnen  of  course 
tne  minimum  value  of  I(p:n)  la  aero  and  tne  minimising  distrioutlon 
la  p^(»)  -  n  (m ) .  More  generally,  tne  minimum  discrimination 
information  tneorem  C 111  etatea  tnat  tne  minimising  dlatrlDutlon 
la  given  oy 


vnere 


(4.5)  M(TiIt,  ,t,)  -  2  axp(T,  Tj  (w)  +T,Tg(»)  +  r.T.  («))  n(») 

and  tne  t»b  are  parameters  wnlcn  are  in  essence  undetermined 
Lagrange  multipliers  wnose  values  are  defined  in  terms  of 
0*,  aj,  ej  oy 

«C  •  ^  in  M(Ti»T.»T®)  “ 

-  (z  exp  {TXTX(»)  +  %Tg(®))T1  (o»)tt(u •))/*(t1»ts.t*) 
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(4.6)  -  JL.  in  M(T,,T,,Ta)  - 

-  (Z  sxpfr^  (*)  +T.T,  (®)  ^aT.(®Jfo(»)n(®8/to(TJ,T>,T§) 

■  <  *  * 

-  (z  «xp  (TjT^w)  +T#^(»)  +  T#T*  (u»)  )Ta  (m)n(m))Ai(rl  .  j 

| 

4 

We  c&n  now  state  a  number  of  consequences  of  tne  preceding. 

We  note  first  tnat  p*(u»)  is  a  member  of  an  exponential 
family  of  distributions  generated  by  tt(ui)  and  as  sucn  nas  tne 
properties  of  members  of  an  exponential  family.  In  particular 
p*(uj)  -  tt(iu)  for  t  •  Ta  -  t#  -  0.  We  may  also  write  (4.4) 

(4.7)  tn  E-llL  m  -  tn  M(t4  ,tb  ,t#)  +tjT1(»)  -h-.T,  (<")  +TaT4(«u) 

tT(uj) 

•  1>  +  tjTj  (uj )  +  t#t;  («)  +  t,T,  (u>) 

witn  L  •  -  tn  M(t1,tb,t#).  Tne  regression  expression  in  (4.7) 
for  *n(p*(ui)A(ui))  wltn  ^(uj),  i;(tt)),  Ta(u»)  as  tne  explanatory 
variables  and  t  ,t#  ,t#  as  tne  regression  coefficients  plays  an 
important  role  lr.  tne  analysis  we  snail  consider. 

We  note  next  tnat  tne  minimum  value  of  tne  discrimination 
information  (4.2)  is 

(4.8)  I(p*:tt)  -  rl  +  tb©J  +  t,9J  -  tn  M(t,  ,t,  ,tb  ) 
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wnere  tne  9*  ’a  ere  defined  in  (4.3)  and  tne  r's  are  determined  to 
satisfy  (4.6).  Using  tne  value  in  (4.7)  it  may  be  snown  tnat  if 
p(t»)  la  any  member  of  tne  family  of  distributions  aatisfying  (4.3), 
tnen 


14.9)  Kpi")  -  I(P»P*)  +  I(p#jn). 

Tne  pytnagorean  property  (4.9)  plays  an  important  role  in  tne 
analysis  of  information  taolea. 

We  note  tnirdly  relations  connecting  tne  9#,s,  tne  t's, 
and  tne  covariance  matrix  of  tne  T(u*)'s.  If  we  define  tne 
matrices  (vectors) 

(d£)'  -  (d«*,  d«£,  08J),  (arj  •  -  (dTx,  or,,  dTa) 

tnen  [11,  p.49] 

(4,10)  (d©#)  -  £*(£r),  (dr)  -  5**l(d©*) 


wnere  £  is  tne  covariance  matrix  of  («),  (u>),  Ta(ui)  for  tne 

distribution  p*  (ui ) ,  tnat  is,  witn 

© 


"  -(T,  («0-e*)  (Tj  (u>)  -9*  )p*(u>),  X*  -  (d*,),!*”1"  (°*U) 


(4.1D  -*r,  . 

»t4  d«j* 


From  (4.5)  it  is  seen  tnat  M(t  ,t#,t#)  is  tne  moment -gene rating 
function  of  T,  («),  T#(o»),  T#  (u»)  under  tne  distribution  n  (<u) ,  nence 
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t ne  ouaul  ant -generating  function  la  given  up  to  quadratic  terns 

«>y 

(4.12)  M(TltTg,Tt)  -  01T1  +  4-  tas  Ot  |  Tj  T| 

wnere 

(4.13)  -  X  ^  (tt>)  it(»),  0tJ  -  I  (a;  (w)  -  9,  )(T4  (»)  -  9,  )  it(u>). 
Thus,  using  (4.12)  In  (4.6),  we  get 

•?  "  «i  +  }  °u  T, 

(4.14)  «*  -  9.  +  2  a,,  t4 

<  -  *•  +  f  T. 

and  tnen  using  (4.14)  m  (4.8)  yields 

(4.15)  21  ( p*  ;it )  *  (9*  -  0)  «  £~l  (e*  -  9)  «  t»  jt. 

We  nave  used  tnree  functions  T,  (<#) ,  T#  (w) ,  T#  (<#)  tnus  far 
in  tne  discussion  merely  as  a  matter  of  convenience.  We  note  tnat 
(t.15)  nolda  for  a  set  of  m  functions  T,  (w) ,  1  «  l,...,m  wltn 
appropriate  meanings  for  tne  matrices.  Let  us  partition  tne  set 
of  m  functions  Tj  (u>)  into  a  set  say  of  m#  and  a  set  H#  of  tne 
remaining  m0  -  m  -  m#  functions,  wnere  tne  functions  In  tne  set 
nave  tne  property  tnat 
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(4.16)  Q*  -  ,  l-l 


We  nave  tne  related  partitioning  of  tne  covariance  matrix  of  tne 
^  (»).  l  -  1,  ...,m 


and  tne  e*,  and  t_  matrices 


(4.18)  e*»  -  «»-  (g*.  g»)f  :•  -  (tJ,  rj). 


In  terms  of  tne  partitionings  in  (4.17)  and  (4.18)  tne 
relations  in  (4.14)  may  oe  written  as 


(4.19) 


£  -  jg,  +2..  I.  +2..  L 


and  using  tne  Tact  tnat  <£  -  ^ ,  it  is  found  tnat  using  tnese 
results  In  (4.8)  now  yields 

(4.20)  21  (p*  jtt )  «  (jg  -  v, ) '  (jjf  -£,)  -  Tj  j§<#4r- 

wnore  2,,..  “2.,  "2,.  2.,  Is  an  m#  x  m#  matrix.  Tne 

results  under  tne  partitioning  will  nelp  in  interpreting  tne 
analysis  of  information  values  and  are  similar  to  tnose  occurring 
In  tne  testing  of  suonypotneses  in  tne  linear  and  multivariate 
linear  nypotnesls  tneory  [11,  p.  216,  259]. 


li 


We  note  from  (4.6)  end  (4.7)  tnat 


(4.21)  -i-  in  (»)  -  \  (w)  «  in  M(t  ,Ta  ,...)  -  t  (u»)  - 

dT,  1  1 

nonce  Tt  (oj)  is  tne  maximum  llaelinood  estimator  of  ©f .  Tnus  If 
we  write  Tt  (*)  «  ^  and  denote  tne  values  satisfying  (4.6)  or 
(4.14)  wltn  ©P  In  place  of  fij*  and  in  place  of  Tf  ,  we  nave 
corresponding  to  (4,15) 


(4.22)  2I(p*.-n)  -22^  ©*  -  2  in  M(?x  ,r,  ,...) 


(e*  -  ©)  •  r1  (©*  ■  «]  »  t_'  i  t_ 


and  corresponding  to  (4.20) 


1 

i 

i 

•i 

-3 

i 

•? 


(4.23)  21  (P*m)  -  &  -  «.)'  Si.,  (it  -  ».)  -i:  2....  i.  • 

A 

We  remara  tnat  tne  covariance  matrix  of  tne  t*s  la  tne  Inverse  of 
tne  covariance  matrix  of  tne  Tt  0»),s* 

A 

If  tne  ©*  are  tne  averages  of  n  Independent  oDservatlons 
tnen  we  nave  for  tne  minimum  discrimination  Information  statistics 

(4.24)  2n  I(p*  ;n)  -  n(©»  -  ©) '  r1  (g*  -  6)  «  n  1*  £  i 
and  in  tne  partitioned  case 

(4.25)  2n  I(p*  :tt)  »  n(£  -  ©.  )  •  ,  ( £  -  «, )  -  n  rj  r,  . 
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A 

Under  tne  null  nypotneele  2n  I(p»m)  in  (4.24)  or  (4.25)  la 
asymptotically  distributed  as  x*  respectively  witn  n  or 
degrees  of  freedom. 

5.  Computational  procedures.  An  experiment  nas  teen  designed 
and  observations  made  resulting  in  a  multidimensional  contingency 
table  witn  tne  desired  classifications  and  categories.  All 
tne  information  tne  experimenter  nopes  to  obtain  from  tne 
experiment  is  contained  in  tne  contingency  table.  In  tne  process 
of  analysis,  tne  aim  is  to  express  tne  observed  table  by  a 
number  of  parameters  depending  on  some  or  all  of  tne  marginals, 
tn&t  is,  to  find  out  now  mucn  of  tnis  total  information  is 
contained  in  a  summary  consisting  of  sets  of  marginals.  Indeed, 
tne  relationsnip  Detween  tne  concept  of  Independence  or  association 
and  Interaction  In  contingency  tables  and  tne  role  tne  marginals 
play  is  evidenced  in  tne  writings  of  Bartlett  Cll,  Simpson  [171, 

Roy  and  Kastenbaum  [161,  Lewis  [151,  Darrocn  [ 2)  and  otners  on 
tne  analysis  of  contingency  tables.  Tnus,  tne  e's  in  tne 
preceding  discussion  will  be  tne  marginals  of  interest. 

5.1.  Tne  TfuQ  functions.  Tne  T(iu)  functions  for  tne  R  x  s  x  T  x  U 
table  turn  out  to  be  a  Daslc  set  of  simple  functions  and  tneir 
various  products.  Tnus,  for  example,  tne  T(w)  function  associated 
witn  tne  one-way  marginal  p(2...)  is 

(5.1)  if(iOKt)  -  1  for  1  -  2,  any 
-  o  otnerwise 
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(5.2)  £  p(ijK^)  S^(ijKt)  -  p(2...). 

Similarly  tne  T(«u)  function  associated  wltn  tne  one-way  marginal 
p(..3*),  for  example,  is 


(5.3)  3f(io*l}-l 

-  0 


for  k  -  3,  any  i,j,4 
otnerwlse 


since 

(5A)  1  p(1jk<,)  tJ(1j«,)  -  P(«  .3.) . 


Tnus  for  tne  r  x  a  x  t  x  u  table  we  nave 


(5.5) 


(r-1) 

(a-1) 

(t-1) 
( o—l) 


linearly  Independent  functions 
linearly  Independent  functions 
linearly  Independent  functions 
linearly  Independent  functions 


lJ(ljK,'),a-l,...fr-l 
T^(ijKt),«  -l,...,s-l 
T^(ioict),Y-l,...,t-l 

T?(ljmt),4-lf...fu-l, 


since,  for  example, 

Z  Z  T^(ijitt)  -  rstu  . 
a  -1 

We  nave  aroltrarly  excluded  tne  functions  corresponding  to 
a  -  r,  *  -  s,  y  “  t,  6  -  u  as  a  matter  of  convenience,  we  could 
nave  selected  a  -  1,  a  -  1,  y  -  1,  •  -  1  or  any  otner  set  of  values. 
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Tne  T{»)  function  Associated  wltn  tne  two-way  marginal 
p(12. . )  say,  Is  ^(Ijm,)  af(ljw)  alnca  from  tne  definition  of 
Tf(ljitt)  and  Tf (ijmt)  it  may  ce  seen  tnat 

(5.6)  Tf(ljRt)  if (ljict)  -  1  for  1  -  1,  3  -  2,  any  *,4 

a  o  otnerwlse 

and 

(5.7)  2  P(ijtt)  Tf(ljR4)  lf(ljR4)  -  P (12. . ) . 

Tnus  tne  T(u>)  function  associated  witft  any  two-way  marginal  la 
a  product  of  two  appropriate  functions  of  tne  set  (5.5) • 

Similarly  tne  T(®)  function  associated  wltn  any  tnree-way 
marginal  will  ce  a  product  of  tnree  of  tne  appropriate  functions 
of  tne  set  (5-5),  for  example, 

(5.8)  2  p(iJK4)  if(ljtt)  tJ^IJK/,)  T^lJKt)  -  p(2.13). 

Similarly  tne  T(®)  function  associated  wltn  any  four-way 
marginal  will  be  a  product  of  four  of  tne  appropriate  functions 
of  tne  set  (5*5).  for  example, 

(5.9)  2  P(ijKl)  T*(ljxO  Tf(lJlM.)  Tj(ijxt)  tJ^Ijw.)  -  p ( 2112)  . 

We  note  tnat  tne re  are  a  total  of 
Nx  -  (r-1)  +  (s-1)  +  (t-1)  +  (u-1) 

N,  -  (r-1)  ( s-l)+ (r-1)  (t-1) +( r-1)  (u-l)+(s-l)  (t-l)+(s-l)  (u-l)+(t-l)(u-3) 
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N,-(r-l) (s-l) (t-l)+(r-l) (8-1) (u-l)+(r-l) (t-1) (u-l)+(s-l) (t-1) (u-1) 
N4-(r-l)(8-l)(t-l)(u-l) 

respectively  of  tfte  eiaple  linearly  independent  functions  end 
tnelr  products  two,  tnree,  four  et  a  tine.  It  nay  be  verified 
tnat 

(5.10)  ratu  -  1  »  N  -  H  +  N#  +  ♦  N4  . 

Tnese  values  are  degrees  of  freedom  in  tne  analysis  of  information 
tables  in  [6],  [101. 

5.2.  Tne  p»(ui)  values.  In  tne  usual  regression  analysis 
procedure,  one  first  computes  tne  regression  coefficients  and 
tnen  gets  tne  values  of  tne  estimates.  In  tnis  case  nowever  we 
reverse  tne  procedure.  Instead  of  trying  to  obtain  tne  values 
of  tne  t * s  from  (4.6)  we  snail  first  obtain  tne  values  of  p#(ui) 
by  a  straigntforward  convergent  Iterative  procedure  and  tnen 
derive  tne  values  of  tne  t's  from  (4.7).  We  snail  not  discuss 
tne  details  of  tne  iteration  nere  since  tney  nave  been  described 
in  [43,  [63,  r9l,  [lol.  Tne  iteration  may  be  described  as 
successively  cycling  tnrougn  adjustments  of  tne  marginals  of 
interest  starting  witn  tne  tt(u>)  distribution  until  a  desired 
accuracy  of  agreement  between  tne  set  of  observed  marginals  of 
Interest  and  tne  computed  marginals  nas  been  attained. 

5.3.  Tne  r  values.  From  tne  definitions  of  tne  T(u>)  functions 
in  section  5.1  it  is  clear  tnat  tney  tane  on  only  tne  values 

0  or  1  for  eacn  value  of  u>.  From  tne  nature  of  tne  T(uj) 

functions  tne  set  of  regression  equations  (4.7)  will  nave  some 
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wltn  &  single  r  value  wnlen  can  be  date mined.  Tnen  tnere  will 
oe  &  set  wltn  one  additional  unknown  value  end  some  of  tne  v's 
already  detemlned.  Tnese  new  unknown  t  values  can  oe  tnen 
determined.  Tnis  process  of  successive  evaluation  is  carried 
on  until  all  tne  values  of  t  are  detemlned. 


6.  Analysis  of  information.  Altnougn  tne  preceding  tneoretlcal 
discussion  nas  oeen  In  terms  of  probabilities,  estimated 
probabilities  or  relative  frequencies,  In  practice  It  nas  been 
found  more  convenient  not  to  divide  everytnlng  by  n,  tne  total 
number  of  occurrences,  and  deal  wltn  observed  or  estimated 
occurrences,  tnat  Is,  wltn  m  (ijat)  -  n/rstu,  x(ljKt),  x(l...), 
x ( . j k. ) ,  x*(ljKt)  -  n  p*(ljKt)  etc.  Tne  analysis  of  Information 
is  oasea  on  tne  fundamental  relation  (4.9)  for  tne  minimum 
discrimination  information  statistics.  Specifically  If  n  p*(w)  - 
x*  (u>)  is  tne  minimum  discrimination  information  estimate 
corresponding  to  a  set  Ha  of  given  marginals  and  x£  (»)  Is  tne 
minimum  discrimination  information  estimate  corresponding  to  a 
set  of  given  marginals,  wnere  ^  c  ^ ,  tnen  tne  basic  relations 


(6.1) 


ZE(x:nrr)  -  2I(3^;m)  +  2I(x:j£) 
2I(x:m)  -  2l(x*:m)  +  2I(xjx*) 
2I(xJ':m)  -  2I(xJ:nn)  +  2I(x£:xJ) 
2X(x;xJ)  -  2l(x*;x;)  +  2l(x:x*) 
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In  terns  of  tne  representation  In  (4.4)  as  an  exponential 
f sally,  for  our  discussion,  tne  two  extreno  oases  are  tne  uniform 
dlstrioutlon  for  wnlon  all  r's  are  zero,  and  tne  ooserved 
contingency  taole  or  dlstrioutlon  for  vnlcn  all  N  -  rstu  -  1 
r's  are  needed. 

Measures  of  tne  form  2l(x:x*),  tn&t  Is,  tne  oonparlson  of 
an  ooserved  contingency  taole  wltn  am  estimated  contingency  table, 
are  called  measures  of  Interaction  and  measures  of  tne  form 
2I(x*:x*),  tnat  is,  tne  comparison  of  two  estimated  contingency 
taoles,  are  called  measures  of  effect,  tnat  is  tne  effect  of  tne 
marginals  in  tne  set  Dut  not  In  tne  set  .  Prom  tne  results 
In  (4.24)  or  (4.25)  we  see  tnat  2I(x:x*)  tests  a  null  nypotnesis 
tnat  tne  set  of  t  parameters  in  tne  representation  of  tne  observed 
contingency  table  x(u>)  Dut.  not  In  tne  representation  of  tne 
estimated  taole  xj(ui)  are  zero,  and  2I(x^sx^)  tests  a  null 
nypotnesis  tnat  tne  additional  set  of  t  parameters  In  tne 
representation  of  tne  estimated  table  x^(«)  but  not  in  tne 
representation  of  tne  estimated  table  x*  (u»)  are  zero. 

Since  tne  marginals  of  tne  estimated  table  x^(ui)  wnlcn 
fora  tne  set  of  restraints  H4  used  to  generate  x^(u.>)  are  tne 
same  as  tne  corresponding  marginals  of  tne  observed  x(u>)  table 
and  all  lower  order  Implied  marginals,  21  (xjxJ^)  Is  al30 
approximately  a  quadratic  in  tne  differences  between  tne 
remaining  marginals  of  tne  x(u>)  taDle  and  tne  corresponding 
ones  as  calculated  from  tne  j£(u>)  taDle. 

Similarly  2I(j^:xJ)  Is  also  approximately  a  quadratic  In 
tne  differences  between  tnose  additional  marginals  in  out  not 

In  Ht  and  tne  corresponding  marginal  values  as  computed  from  tne 
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X*(w)  t&Dla. 

As  wo  snail  see,  because  of  tne  nature  of  tne  T(w)  funotlons 
described  In  section  5.1,  tne  t’a  are  determined  from  tne 
regression  equations  (4.7)  as  sums  and  differences  of  values  of 
An  x*(lomf.).  A  variety  of  statistics  nave  been  presented  In 
tne  literature  for  tne  analysis  of  contingency  tables  wnicn  are 
quadratics  In  tne  marginal  values  or  quadratics  In  tne 
logarithms  of  tne  observed  or  estimated  values.  Tne  principle 
of  minimum  discrimination  Information  estimation  and  its 
procedures  tnus  provides  a  unifying  relationship  since  suen 
statistics  may  oe  seen  as  opposite  faces  of  tne  minimum 
discrimination  information  statistic. 

We  nave  presented  tne  approximations  in  terms  of  quadratic 
forms  in  tne  marginals  or  tne  t*s  to  assist  in  understanding  and 
interpreting  tne  analysis  of  information  tables  as  a  bridge 
connecting  tne  familiar  procedures  of  classical  regression 
analysis  and  tne  procedures  proposed  nere.  Tne  covariance  matrix 
of  tne  T(<u)  functions  can  De  estimated  for  eltner  tne  observed 
table  or  any  of  tne  estimated  tables  and  tne  Inverse  of  tnat 
matrix  found  snould  tnelr  values  oe  desired. 

7.  Tne  2xg  table.  Before  we  present  an  application  of  tne 
preceding  Ideas  to  experimental  data  In  a  four-way  contingency 
table,  we  snail  reexamine  tne  2x2  table  from  tne  point  of  view 
cf  tnis  paper.  Tne  algeDraic  details  are  simple  In  tnis  case  and 
exnlblt  tne  unification  of  tne  information  tneoretlc  development. 

Suppose  we  nave  tne  observed  2x2  table  in  figure  7*1» 
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If  *e  fit  tne  one-way 


Figure  7*1 

marginals,  tne  generalised  Independence  nypotnesis  Is  tne 
classical  Independence  nypotnesls  and  tne  minimum  discrimination 
information  estimate  is  3^(lj)  -  x(i.)x(.j)  /n.  A  convenient 
representation  of  tne  regression  (4.7)  is  given  in  figure  7.2. 
Tne  entries  in  tne  columns  rx  ,  r9  ,  r. 


1  j  ||  L  t.  T,  r. 


Figure  7.2 


are  respectively  tne  values  of  tne  functions  Tx (ij) ,  1J»  (ij) ,  T,  (lj) 
associated  wltn  tne  marginals  9X  -  x(l.),  9^  •  x(.l),  9*  -  x(ll), 
and  tne  column  needed  L  corresponds  to  tne  negative  of  tne 
logarltnm  of  tne  moment -gene rating  function.  For  tne  observed 
dlstrioutlon,  recalling  tne  regression  (4.7),  It  is  found  tnat 
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(7.1)  t  -  in  (x(22)AA),  Tt  -  *n  (x(12)A(22)),  t,  -  *n  (x(21)/x(22)) 

%  -  in  (x(li)x(aa) A(i2)*(2i) ) . 


If  we  call  £  tne  matrix  wltn  ontumne  tM  oolumaa  of  Figure  7.2, 
tnat  is, 


(7.2)  S 


•(i 


i  i 
1  o 
o  1 
0  0 


and  daflna  a  diagonal  matrix  D  wltn  main  diagonal  tna  elements 
x(lj),  tnat  la. 


(7.3) 


*■( 


X(ll)  0  0  0  \ 

0  x(12)  0  0  1 

0  0  x(2l)  0  I 

.  0  0  0  x  ( 22)  / 


tnen  It  may  oe  verified  tnat  tne  eatlmate  of  tne  covariance  matrix 
of  tne  ^  («)  for  tna  ooeerved' contingency  tacle  la  £  -  wnere 


\4)  A  -/ |  -  T'DT 

-  \S>  A ,.)  " 


(7-5)  -S.  -  A..  Si  A,. 


and^,i8lx  1,  la  3  x  3,  -  4,  la  1  x  3.  It  la  found  tnat 


(7.6)  i- 


x(ll)  - 


x(U)  - 


xfi.lxf.: 


n 


x(ll)  -  *LHiL 


Sven  for  tnis  ample  case  inverting  tne  matrix  in  (7*6)  is  messy 
algeoraioally,  nowever.it  is  easier  to  use  tne  relations  in  (4.10) 
"and  (4.11) .  Vs  nave  from  (7*1) 


(7.7)  t4-  x(12)  -<tnx(22)#  In  x(21)  -tnx(22), 

t,»  In  x(ll)  ♦  4n  x(22)  -  In  x(12)  -  4n  x(21) 


and  from  8^  -  x(l.)f  ^  -  x(.l),  4^  -  x(ll)  and  tne  relatione 
implied  in  Figure  7.1  it  is  found  tnat 

(7.8)  x(ll)  -  x(12)  -  ^  x(21)  -  q,-^,x(22)-n-^-q,  +  * 

It  tnen  follows  tnat 


(7.9) 


il*. . 

1  + 

1 

All.  . 

1 

*(12) 

x(22)  * 

ao. 

x(22) 

Ala.  . 

1 

x(22)  * 

Ala.  . 

a®. 

l 

*(21)  + 

1 

x(22) 

1 

1 

"  *(12) 

”  x(22) 

X 

Ala.  .  . 


1 

x(22) 

1 

x(22) 


^  1 


&<?,  x(ll)  x(12)  x(21)  x(22) 


tnat  is,  tne  entries  of  zf1  since  Al*.  »  o‘ < 

'  *  A 
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Hot*  tnat  tna  value  of  tns  logarltnm  of  tns  cross-produot 
ratio  as  a  measure  of  association  appears  in  tna  course  of  tna 
analysis  as  tna  valua  of  t#,  and  tnat  %  •  0  for  x*(lj)  wnoaa 
raprasantatlon  as  In  Figure  7.2  does  not  involve  tna  last 
column.  Tna  minimum  also rinlnat ion  information  statistic  to 
test  tna  null  nypotnesls  of  Independence  is  8X(xix*).  Xn  tnls 
oasa  «£  m  %•  fif  -  \  and  In  accordance  vitn  (4.25) 


(7.10)  2I(x:x») 


(x(ii)  -  *.fcOxL2I  )• 


Remembering  tnat  x*(ij)  -  x(l.)x(  .j)/n,  tna  rignt-nand  side  of 

(7.10)  may  also  os  snown  to  Da 

(7.11)  2  (x(l z)  -  x(i.)x(.j)/n)a/ iiidiLil 

n 

tne  classical  X*-test  for  independence  vltn  one  degree  of  freedom. 
A  test  wiicn  nas  Been  proposed  for  tna  null  nypotnesls  of  no 
association  or  no  interaction  in  tna  2x2  taole  is 

(T.12)  (<.n  *(U)Un  x(22)-tn*(12)-te  x(21  ))* 

wnlcn  is  seen  to  oe  tne  approximation  for  2I(x:x*)  in  terms  of  tne 
t's  vitn  tne  covariance  matrix  estimated  using  tne  ooserved  values 
and  not  tne  estimated  values.  We  remarx  tnat  if  tne  observed 
values  are  used  to  estimate  tne  covariance  matrix  tnen  instead 
of  tne  classical  Xa-  test  in  (7.11)  tnere  is  derived  tne  modified 
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Neyman  cni-square 


(7.13)  X»*  -  l(x(lj)  -  x(i.)x(.j)/h)*/x(lj) . 

8.  jgSSg&Si,  wltft  experimental  data.  consider  tne  B«  3  n  in  II 
table  8.1a  representing  tne  results  of  test  snooting  under  tnroe 
different  conditions: 

R:Oun  barrel  wear: 1-1,  new,  1-2,  moderate,  1-3,  excessive 

S:Gun  barrel  temperature : j— 1,  cold,  j-2,  not 

T:Unit  temperatureiK-1,  not,  tc-2,  ambient,  k-3,  cold 

UtNumoer  operative jt-1,  success,  t-2,  failure. 

We  are  indebted  to  Mr.  B.M.  Kurxjian  of  tne  Harry  Diamond 

Laboratories  for  tne  data  and  nls  Interest  In  tne  analytic 

procedure  we  nave  discussed.  We  note  tnat  13  rounds  eacn  were 

fired  under  eaon  of  18  experimented  conditions.  Tnls  la  not 

necessary  for  tne  application  of  tne  analysis  of  Information 

procedures  out  was  required  for  tne  earlier  application  of 

Brandt's  analysis  to  tne  data. 

Figure  8.1  presents  a  grapnlc  representation  of  tne 

regression  (4.7)  and  Is  similar  to  tnat  In  Figure  7.2  for  tne 

2x2  table.  Tne  L  column  corresponds  to  tne  negative  of  tne 

logarithm  of  tne  moment-generating  function  (a  normalizing  value) 

and  eacn  of  tne  otner  oolumns  is  a  T(u>)  function  wltn  tne  associated 

r  value  at  tne  nead  of  tne  column.  Superscripts  and  subscripts 

are  used  to  Identify  tne  factors  and  categories  Involved.  Tne 

complete  representation  In  Figure  8.1  witn  tne  35  T  values  will 

provide  an  exact  representation  for  tne  observed  values  x(u>)c 

Tables  8.2,  8.3,  and  8.4  are  analysis  of  Information  taoles 
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presenting  appropriate  analyses  as  various  sets  of  marginals  of 
-interest  Are  introduced  as  explanatory  varieties. 

In  Figure  8.2  tne  columns  corresponding  to  tne  T  parameters 
wnlon  enter  Into  tne  various  distributions  appearing  In  tables 
8.2,  8.3  end  8.4  nave  been  oneoxed,  Mote  tnat  for  m,  tne 
unifora  distribution,  tnere  are  no  cneoxs,  and  for  x(«),  tne 
observed  distribution  all  columns  are  cnecxed.  Tne  degrees  of 
freedom  for  any  effect  component  Is  tne  difference  In  tne 
number  of  columns  cnecxed  for  tne  corresponding  estimates.  Tne 
degrees  of  freedom  for  any  Interaction  component  Is  tne  difference 
In  tne  number  of  columns  cnecxed  for  tne  observed  x-dlatrlbutlon 
and  tne  estimated  distribution. 

Tne  null  nypotnesls  for  any  effect  component  la  tnat  tne 
additional  t  parameters  are  zero,  for  example,  tne  nrll 
nypotnesls  for  tne  effect  component  21  (x£ :x£)  In  table  8.2  is 
tnat  t^,  are  zero.  Tne  null  nypotnesls  for  any  Interaction 
component  Is  tnat  tne  set  of  parameters  wnlcn  are  crecxed  for  tne 
observed  x-dlstrlbutlon  but  not  for  tne  estimated  distribution 


are  zero,  for  example,  tne  null  nypotnesls  for  tne  tnlrd-order 

interaction  component  2l(x:x*)  In  taDle  8.2  Is  tnat 
_RSTU  „RSTU  _RSTU  Mro. 


t.T.Y  are  Z8ro 


Note  tnat  all  tne  marginals  implied  for  xj  in  table  8.4 
are  x(l...),  x(.j..),  x(..x.),  x(...t),  x(lj.*),  x(l.x.),  x(l..t), 
x( . jx. ) ,  x(.j.t),  x ( 1 jx. ) |  x ( 1 j . 4 )  and  tne  marginals  Implied  for 


In  table  8.4  are  x{l...),  x(.j.«),  x(..k«),  x(...t),  x  ( 1  j .  • ) , 
x(i.x.)  ,  x(l..t)  ,x(.0X.)  ,x(  .J.t)  ,x(.  .xt)  ,x(ljx.)  ,x(lj./,),x(l.xO  , 
nance  tne  six  parameters  T ^ , t TU ^ T FTU f T  RTU f t^’ 1 , t  appear  In 

x*  but  not  In  x*. 


25 


We  draw  tne  following  conclusions  from  tables  8.2,  8.3,  8.4: 

1.  Success/Failure  is  not  nomogeneous  over  tne  18 

experimental  situations,  -  x(tjK.  )x(  ...i)  Ai,  2I(x:xf)  - 

34.371,  17  D.F. 

2.  Tne  effect  of  x(i..4)  In  table  8.2  Is  almost  significant, 
but  tnose  of  x(.j.i),  x(..xt)  are  not  significant,  nence  we 
proceed  as  in  table  8.4. 

3.  Tne  marginals  x(ijtc.),  x(l,).t),  x(i.xt)  and  tne  lower 
order  marginals  tney  Imply  provide  an  acceptable  estimate  for  tne 
original  data  since  2I(x:x*')  -  7.413,  6  D.F.,  tnat  is,  we  accept 

a  null  nypotnesis  tnat  tne  set  of  six  parameters  t^TU 

,RSTU  RSTU  RSTU  RSTU 
im»Taiii»  Tim  »  T«i»i  are  zero* 

4.  Using  Figure  8.1  and  Figure  8.2  we  can  express  tne 

logaritnm  of  tne  ratio  of  tne  estimates  for  success  to  failure 
under  all  18  experimental  conditions,  tnat  is,  tne  logit,  as 
tne  linear  combination  of  a  constant  term  t”,  a  term  depending 
on  Darrel  wear  t^,  a  term  depending  on  tne  interaction 

of  barrel  wear  and  barrel  temperature  and  a  term 

depending  on  tne  interaction  of  barrel  wear  and  unit  temperature 
tRTU  _RTU  -rtu  „RTU 

111  »  T»ll  »  Tl»l »  t8B1  • 

in  ♦  rff  *  tf"  ♦  T™  *  ,JS«  .  t.BTO 

x*  (1112)  1  1  14 

.  x*  (1211)  U  RU  TTJ  RTU 

in  44 -  Tx  +  tix  +  tx1  +T.1X 
X*'(1212)  4  4  4  4  4  44  4 

tn  +  Tfxu  ♦  t™  +  rffy  ♦  rj™ 

<(2112) 

In  -  T?  +  T™  ♦  T™  +  T™ 

xj (2212)  *44 
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*n  £13131 i 

<  (3132) 

tn  £13231) 

<(3232) 


TU  +  TSU 
T1  +  Tli 


-  T? 


5.  Since  tne  computer  program  provides  not  only  tne  values 
c f  <(ijKt)  out  also  tne  values  of  tn  (<(ijKt)/< (3232) ) ,  tne 
values  of  tne  T’a  in  conclusion  4  above  can  be  easily  found. 

T;J  -  tn  £Jl£3*l  _  3.0281 
<(3232) 

-  tn  .£  j1^31,)  -  Ty  .  -  1.6470 
<(1232)  1 

rjf  -tn  £12212.  2.2870 

<(2232)  1 

Tf!r  -  tn  -  tU  -  -  0.6794 

<(3132)  1 

T™  *  *»  4^4  -  -  1-9759 

<(3212)  1 


TU 

T»i  “  *n 


£,(3,221)  ,  tU  _  0> 


<  (3222) 


7746 


rRSU 

111 


RSU 
r»i  1 


RTU 

rm 


RTU 

r»n 


-  tn  £11131)  .  U  -  ™  ,  W  »-q.  2928 

<(1132)  1  11  11 

-  tn  -  <  -  t*»  -  r?J  -  1.7215 

X.  (2132)  11 

f  <(1211)  U  RU  TU  „ 

n  ^TuSIST  T‘  'T“  'T“  2-3336 

.  *»  5!22L  -  T»  -  T».  T™  -  i.4528 

<(2212)  1  11  11 
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0.1639 


£ 

m. 

v 

*•- 

r 


mu  .  tn  iflSJ. .  Ty  .  w  .  tu  .  0.1635 

X*(1222)  11  11 

X*  (2221)  _  _U  _EU  _  _TU  _  n  e0^fi 

*•*  n  T»  T"  T“  "  0,5878 

A»  a  cnecx  we  nave,  for  example,  Ln  (x* (1111) /x*  (1112))  - 
0.7666  ana  ,»  .  t™  +  t??  *  t™  ♦  T»»  *  T«f®  .  0.T666. 

6.  Tne  values  of  L  and  otner  t  parameters  for  tne  x£- 

distribution  can  De  obtained  from  Figure  8.1  and  tne  computer 
Hating  of  tne  values  of  tn(xj  (ijKtJ/x*' (3232))  and  tn(x?  (3232)/^) , 
in  tnls  case  nrr  -  270/(3*2x3x2).  Tnus  L  -  -2.3822, 

tJ  -  tn(xj (1232) /x*( 3232))  -  1.4701,  etc. 

7.  Tne  computer  output  for  x^(ijKt)  is  listed  aB  taole  8.5. 
Five  values  are  given  for  eacn  l,j,K,t,  tneae  are: 

Observed:  x(ijicl) 

Predicted:  xJ(ljKt) 

Residual:  x(ijnt)  -  x^ijKl) 

Standardize:  2  x(ij&t)  tn  (x(lj*t)/x£ (ijket) ) 

Log  ratio:  tn(x£ (ljxt)/xJ  (3232) ) . 

Tnere  is  also  given  tne  value  of  2I(x:x*)  along  wltn  tne  degrees 
of  freedom  and  a  probability  Daeed  on  tne  x*-  distribution  and 
tne  value  of  L  as  log(x  STAR/N/CELLS) . 

9.  Ac unowle dgnent .  Tne  Interest  and  cooperation  of  Professor 
C.T.  Ireland  and  Dr.  H.H.  Ku  are  gratefully  acknowledged. 
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...  Iheniii-I  nu k. a., . ^  1,  •,>.  . jh 


TaDle  8.1a  Original  Data  x(ljKt) 


•wir f*r 


W  f'hAl-  <m*A  * 


Component  due  to 
a)x(iotc.) 

o)x(l0K.)  ,x(...t)  “ 

U-effect 
interaction 

c) x(ijx.),x(i..t)  '"—~ 

RU-effect  JRST 
Interaction 

d) x(ijK.)  ,x(i.  .t)  ,x(7j."t] 

SU-effect.  |  RST ,  RU 
Interaction 

e) x(ijx.) ,x(i.  t) ,x (7j  . I ) , x ( . . Kt ) 

TU-efrect|RST,RU,SU 

Interaction 

f) x(ijK.) ,X(. .ftt) ,x(ij.t) 

RSU-effect |RST, RU,SU,TU 
Interaction 


Analysis  of  Information 
Table  8.8 

Information 


2I(x:x*)-99,639 

2I(xJ:xJ)-65.268 
2I(x:xj»)  -34.371 

2I(xJ:x*)-  5.303 
£I(x:x* )  -29.068 

21 (x* :x»)-  0.314 
2I(x:xf)  -28.754 

2l(x*t:x*)-  2.705 
2I(x:x*)  -26.049 

2I(x*:x*#)-  9.752 
2I(x:x*f)  -16. 297 


g)x(ljK.) ,x(ij.t) ,x(I.Kl) 

RTU-effect I RST, RU,SU,TUs  RSU  21  (jf  ;x*)-  8.89I 
Interaction  2I(x:xf)  -  7.4o6 


n)x(ijK.),x(ij.t),x(i.Kt),x(.jKt) 

STU-effect  !RST,RU,SU,TU(RSi;RTU2I(xJ:X^)-  4.543 
Tnird-order  interaction  2I(x:xJ)  -  2.863 


.*  — 


- 
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Table  8.3 


Component  due  to 

Information 

D.P. 

d)x(lJK.)  ,X(I.  .t  )  tx(.  J.t) 

2I(x:xf )  .  88.754 

14 

a}x(ljx.),x(ij.4) 

RSU-effectj RST,RU,SU 

2I(j£uf )  -  9.649 

8 

interaction 

2I(x«x* )  -  19.105 

12 

fJx(lJK.),X(10.t),X(..Kt) 

TU-effect|  RST,RU,SU,RSU 

2I(x*:x?)  •  2.8o8 

2 

Interaction 

2I(x:x*)  -  16.297 

10 

Table  8.4 

Component  due  to 

Infonnation 

D.P. 

D)x(iju.),x(...t) 

2I(x:x*)  -  34.371 

17 

c)x(lon.),x(I..t) 

RU-ef fectl  RST 

2I(xJ:x*)  -  5.303 

2 

Interaction 

2I(x:x*)  -  29.068 

15 

m)x(ljx.)  ,x(lj.t) 

RSU-effectj  RST,RU 

2I(Xj!X*)  -  9.963 

3 

Interaction 

2I(x:j£)  -  19.105 

12 

n)x(ljK.),x(lo.t)  ,x(l.ttt) 

RTU-ef fectj  RST, RU, RSU 

2I(j£:j£)  -  11.699 

6 

Interaction 

2I(x:j£)  -  7.406 

6 

X*  *r  x* 
■  • 
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tu-vs 


- RtStCLUM 


T*ol«  8.5 

TPayput'**  'Output' 


AUflLSQJjLeX5J 


1  DflSERVfcO 

_ 1 _ J»K£0lCTk0_ 

1  RESIDUAL 
_ 1_$IAN0ARUIZE _ 

1  LOG  RATIO 

2  _ OBSERVEO _ 

2  PKtOIC  T£0 

_ 2 _  RESIDUAL _ 

2  STANDARDIZE 

_ 2 _ LOG  RATIO _ 

3  OBSERVEO 

_  3 _  PKtCICTEQ _ 

3  RESIDUAL 

_ 3_$JA  N  0£R0 1 1 E _ 

3  LOG  RATIO 


9.00COOO 

__10. 241535  _ 

•1.24  1535 
_Z.l,  163043 _ 

_ _ 

_ B.OOCOOO _ 

6.746210 

_ 1.253790  _ 

1.36)679 

_ Z±2Uttl _ 

9.0OCOOO 

_  9.012303 _ 

-0.012303 
_rO. 012295  __ 
_ 2.S639U3 _ 


6.0O0C00 

_ 4.7S8C02 

1  .241990 

_ 1  .391587 

r.9??i<rP 

_ 7.000C00 

8.254419 

_ -1.254419 

-1  .153870 

_ 2.U2fl£6i. 

6.000COO 

_ 5  .987520 

0.012480 

_ 0.012490, 

. _ LtllUil £ 


RESIOUALS I  R  *  S  *  T  * 


FIRST 


2  SIBSCRXPT Sx 


l 

OBSERVEO 

1 _ 

14.COCOOO 

1 .OOOCOO 

1 

PREOICTEO 

2 

12.75E455 

2.241597 

.1. 

RESIDUAL _ 

3 

.1.24  1545  _ 

-1 .241597 

1 

STANDARDIZE 

4 

1.3GC076 

—0 .807  267 

1 

LOG  RATIO 

s 

n  — 2,312506 _ 

_ Ulliiia 

2 

OBSERVED 

6  “ 

9.00C000 

6. OOOCOO 

2 

PREOICTEO 

7 

1C. 252795 

4.745572 

2 

*  RESIDUAL 

a 

- 1. 25  2795 

*  1.254428 

-*.173809 


1  .407280 


2 

LUG  RATIO 

10 

2,694960 

_ 1  AUilZZ. 

3 

OBSERVtO 

11 

12.00C00O 

3,  000 COO 

3 

PREDICTED 

12 

1  1.967688 

3  .012478 

3 

residual 

13 

0.012312 

-0.012478 

3 

STANDARDIZE 

14 

0.012308 

-0.012452 

3 

LUG  RATIO 

15 

- 2.B51192 _ _ 

_ Ui2fl£I2 

RESIOUALS >  R *  S  «  T  *  U.  FIRST 


2  SIBSCRIPTSi 
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1 _ OBSERVfcO _ l _ 1  l. OOOOOO _ 6 .OOOCOO. 

1  PRE  CICTfcO  2  1 1*66  5621  3.314257 

V _  RESIDUAL _ 3 _ >0.683621  0. 6837*3 

“l  STANDARDIZE  A  -0.665103  0,752261 

_1 LOO  RATIO _ 3 - 1~B  ?  flftlfl— ,  1^365363 

2  OBSERVED  6  14. OOCOOO  1  .OOOCOO 

2 _ PREDICTED  7 _ 12*672381 _ 2*527138. 

”*2  RESIDUAL  8  1.527119  -1.527  1SB 

2  StANOAROl<| _ 0 _  1.617001 _ -0.927C95. 

~i~  LOO  RATIO  TO  - _~2*fl9C36a--  . 1*2.93802 

3  UB SERVED  11  12.000000  _3 ,000000 

■3  PR EDICT £0  12  12.861680  2,158596 

3  RESIOUAL  13  -0.86  1680 _ 0.86 1 606 

3  STANDARDIZE  16  -0.812287  0,987665 

^  _ 3_ _ 1  nr.  RATIO  IS  _ 2*91593* - 


R E S 1 0 UAL S 1  R  ♦  S  »  T  *  L.  FIRST  _ 2  SUBSCRIPTS*  .  2 


2 


1 

l 

OBSERVED 

PREDICTED 

1 

2 

9. OOCOOO 
8.316383 

6. OOOCOO 
6.685740 

1 

RESIDUAL 

3 

0.685617 

-0.685  740 

L  STANDARDIZE 

4 

C. 717138 

-0.649205 

1 

log  ratio 

5 _ _ 

_ ii.,i612£S 

2 

OBSERVED 

6 

8. OOCOOO 

7. OOOCOO 

2 

PREOICTtO 

7 

9.527126 

5.472640 

2 

residual 

8 

-1.527126 

1  .527  160 

2  standardize 

9 

-1.357613 

1  .722786 

2 

log  ratio 

10—— 

_ i»6maa— . 

_ 2aQ8IIIQ 

3 

OBSERVED 

11 

1 1. OOCOOO 

4. OOOCOO 

3 

PREOICTtO 

12 

1C. 156509 

4.841  <.04 

3 

RESIDUAL 

13 

0.84  1491 

-0.841404 

3  : 

STANDARDIZE 

16 

C. 875411 

—0 .763642 

3 

LUG  RATIO 

15 _ 

_ Uk&itlZA— , 

_ U215516 

residuals*  r  *  s  * 

T  *  U. 

FIRST  2  SUBSCRIPTS*  3 

1 _ 2 


1 

OBSERVED 

l 

9. OOCOOO 

6. 000 COO 

1 

PREDICTED 

2 

8.882741 

6 .110252 

l 

RESIOUAL 

3 

C. 117259 

-0.118252 

1 

STANDARDIZE 

4 

0.1ie029 

-0.117102 

1 

LOG  RAT  JO 

5 _ 

Ui  5.1hZZ _ 

_ _ 2^iiasufl 

2 

OBSERVkO 

b~ 

"l  }. OOCOOO 

2. OOOCOO 

2 

PREOICTtO 

7 

12.425370 

2.574204 

2 

RESIOUAL 

8 

0.574630 

-0.574  2C4 

2 

STANDARDIZE 

9 

C. 587705 

-0.504 664 

2 

LOG  RATIO 

10 _ 

_ _ 

_ 

3 

OMMVED 

11 

13.000000 

2.000000 

34 


f 

r 


r 

?. . 

.  .3  .PREDICTED 

12 _ 

13.691921 

.  .1.307411 

i 

r 

3  RESIOUAL 

13 

-0.691921 

0.692589 

l _ 

J-JklANUARAUS _ 

1R_ 

-0.674136 

_ 0.630196 

k 

% 

l - 

31 LOG  RATI  (3^ 

1» 

..2*242114- 

_ 5*515141 

t 


|  RESIDUALS*  R  *  S  *  T  *  U.  FIRST  2  SlBSCRIPTSi  3 


2 


; _ l _  OBSERVED _ V _ .ll.OOCOOO _ A  *000 coo _ 

1  PREDICTED  2  11.117260  3.861743 

_ 1  RESIDUAL _ 3 _ _-C.  117260 _ 0.118257  _ 

1  STANDARDIZE  4  -C. 116640  0.120C37 

I _ l LOG  RATIO _ s _ 2*221fllfl _ 1*222522 -  - 

i  2  OBSERVED  6  li.OOCOOO  Z.OOOCOO 

: _ 2 _ PREDICTED _ 7 _ 1  3.57462  2  _ 1 .42S6VS _ 

2  RESIDUAL  8  -C. 574622  0.574305 

_ 2  STANDARDIZE _  9 _ -0. 5^2285  0.676574 _ 

'  2*  LOG  RATIO  10 _ 2*225512 _ 0*221522 

: _ J  OBSERVEO _ 11 _ l  5.  OOCOOO _  _0.0D0C05  _  . . 

;  3  PREUlCtEO  12  14.308073  0.692593 

! _ 3 _ RESIOUAL _ 13 _  C.691927  _-0. 692588 _ 

3  STANDARD I ZE  14  0.708390  -O.OQOC59 

_ 3 _ I.CilRATIQ _ 15 _ 3*02iliS _ S0*000ifil _ _ 


HVP0TH6 S IS  V  21  IRIX*)  -  7.406  DEGREES  ~oV*~ FKEEDUM  ' 

6  PROBABILITY  UP  A  LARGER  VALUE  •  0.284956 

LOG (X STAR /N/CELLS)  - 


-2.382215 
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FIELD  TESTING! 

METHODOLOGICAL  CONSIDERATIONS  AND  A 
SPECIFIC  EXAMPLE 


| 

i 

i 


s 

t 

i 

? 

e*. 

$ 

t 

? 

I 

I 


f 


* 

T.  S,  Donaldson  and  R.  J.  Kaplan 
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I.  INTRODUCTION 

Tha  graduate-achool-trained  axparlaantallat  who  goaa  out  into  tha  world 
r‘  of  industry  or  govarneant  aarvlca  ganarally  rune  into  a  aevara  period  of 
intellectual  and  smotlonal  itraaa.  At  flrat  ha  will  try  to  apply  hla  edu¬ 
cation  directly  by  conalderlng  tha  world  to  ba  juat  another  laboratory 
where,  with  the  application  of  proper  controls, he  can  run  exparlmanta  juat 
Ilka  the  onea  ha  waa  uaed  to  doing  In  school  or  reading  about  In  tha 
publiahed  literature.  It  aoon  becoeea  apparent  to  him  that  he  cannot,  and 
that  the  world  la  just  coo  dirty  a  place  to  accommodate  his  pure  way  of 
thinking.  The  need  to  do  something  constructive  for  his  employeer,  however, 
quickly  becomes  a  pressing  matter,  and  at  this  stags  his  attitude  la  one  of 
resignation  to  doing  less  th*r  flrsc-clasa  work,  He  thinks  you  can  go  out 
intc  the  world  and  observe  things,  even  measure  a  Pew  of  the  significant 
parameters  with  s  reasonable  degree  of  accuracy,  but  that  there  Is  r.o 
chance  of  exerting  any  substantial  control  over  tha  altuaticn.  Any  con¬ 
clusions  you  draw  from  such  activities  are  the  result  of  good  Intuition  or 
luck  an  not  by  virtue  of  any  rigorous  application  of  scientific  methodology 

Moat  of  us  remain  in  that  last  stage,  a  fact  which  is  both  unfortunate 
and  unnecessary.  Our  major  point  in  this  paper  is  that,  vMlf>  It  is  Indeed 
true  that  tha  world  out  there  is  dirty  and  confused  compared  to  our  labora¬ 
tory,  we  can  do  experiments  in  it.  There  are  methods  available  to  the 
sophisticated  researcher  which  allow  him  both  to  exert  a  great  deal  of 
control  over  Investigations  taking  place  in  a  natural  environment  and  to 
draw  valid  conclusions  from  his  observations.  Before  discussing  a  specific 
example,  1st  us  look  at  some  of  the  characteristics  of  what  it  Is  that  ws 
can  do  In  the  field  to  further  the  goals  of  our  various  establishments. 


The  first  thing  to  be  made  clear  is  that  there  Is  not  a  dichotomy, 
but  rather  a  continuum  between  laboratory  experimentation  and  field 
testing.  The  world  is  dirty  to  different  extents  and  therefore  the 
degree  of  control  that  we  can  exert  in  our  tests  will  be  different. 

Ghat  must  be  thought  out  very  carefully  before  any  test  is  started  is  the 

Any  views  expressed  in  this  paper  are  those  of  the  author.  They  should 
not  be  Interpreted  as  reflecting  the  views  of  the  RAND  Corporation  or  the 
official  opinion  or  policy  of  any  of  its  governmental  or  private  research 
sponsors.  Papers  ate  reproduced  by  the  RAND  Corporation  ar  a  courtesy  to 
members  of  its  staff. 

The  remainder  of  this  article  was  reproduced  photographically  from  the 
author ‘a  manuacrlpt. 
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tradeoff  bitvtin  what  it  it  we  have  to  have  found  out  whan  we  have 
completed  the  work  and  the  rigor  with  which  we  conduct  the  research. 

The  most  carefully  done  and  acientif Ically  elegant  study  which  answers 
totally  uninteresting  question*  is  as  useless  to  us  as  the  collection 
of  poor  and  Insufficient  evidence  about  crucial  topics.  It  Is  the  task 
of  the  researcher  in  the  field  to  find  the  happy  medium  between  these 
equally  undesirable  extremes. 

The  different  purposes  for  which  we  might  want  to  conduct  a  field 
test  must  also  be  kept  in  mind,  At  one  end  of  the  spectrum  is  the 
notion  of  evaluation.  The  question  to  be  answered  here  has  to  do  with 
whether  or  not  the  equipment  or  system  lives  up  to  the  expectation  of 
its  design  specifications,  and  In  this  case  the  degree  of  unusual 
circumstances  in  which  the  system  can  be  made  to  operate  will  be  a 
determining  factor  in  che  utility  of  the  results.  The  other  end  of 
the  scale  can  be  loosely  defined  by  something  called  policy  formulation. 
Haro  ws  slrssdy  have  en  operating  system  end  we  era  trying  to  learn 
aomethlng  about  Its  operstlonal  impact  both  on  Its  own  operating 
aff Iciency  and  on  tha  interactions  It  hss  with  the  rest  of  the  orga¬ 
nisation.  The  teat  here  is  not  necessarily  limited  to  the  system 
itself  which  Is  under  scrutiny  as,  for  example,  in  s  test  to  decide 
which  of  e  number  of  different  deployments  of  s  new  armored  vehicle 
would  be  best  suited  to  the  accompanying  Infantry.  Tha  test  we  will 
usa  as  a  sample  case  latar  In  this  discusoion  is  of  the  policy- 
formulation  typa,  and  we  hope  it  will  expose  some1  of  the  intricacies 
of  f leld-teatlng  for  this  purpose. 

The  prime  requirement  laid  on  tha  researcher  who  enters  cha 
field-testing  business  1*  that  of  being  imaginative.  In  this  snvlron- 
ment  It  Is  not  possible  to  follow  the  "cookbook"  techniques  which  can 
assure  success  In  most  graduate  achools.  Vou  cannet  change  the 
problem  to  meet  the  (pacifications  of  your  laoornccry  and  you  must, 
therefore,  be  able  to  adept  and  band  your  methods  of  investigating 
and  the  tools  available  to  you  to  meat  the  task  before  you  with  varying 
decrees  of  exact Itud-;.  The  matching  of  procedures  and  methods  to  a 
specific  case  must  be  done  with  full  knowledge  of  what  has  been  given 
up  end  what  has  bean  gained  along  the  way.  Nothing  can  replace  the 
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creativity  of  the  reaearcher  »<  s  ">*  method  must  be  workod  out  for 
each  case  and  cannot  be  taugh  v  r  all.  He  must  be  able  to 

keep  the  whole  problem  In  alno .  .>  the  epeclflc  detail*  together 

with  the  conproalaee  which  wuat  b*  u~..  ,  and  be  conetantly  alert  to 
the  course  of  collecting  data. 

Thi*  bring*  us  to  the  l eat  point  wo  want  to  make  about  field 
■'teat*  in  general  before  getting  on  to  a  specific  caat  In  ordor  to  focua 
our  discussion,  and  rtist  la  aarendlpity.  Plaid  tearing  la  done  ns- 
csssstliy  in  a  very  rich  environment;  too  rich,  in  fact,  end  rhet  ia 
what  causae  all  the  prob lead  rotating  to  lnaddguato  control#.  We  can 
be  on  the  lookout,  however,  for  Instances  where  we  can  turn  this  rich* 
nase  to  our  advantage,  Kvrphey’s  taw  never  fell*  to  operete  in  e 
field  tea1: --the  unexpected  will  always  happen.  8oaetlSMS  it  will  be 
dlaaetoroua  and  wo  will  bo  lucky  to  salvage  anything  froai  the  data 
oe  have  ao  laborously  collected,  Other  kt«es,  however,  we  will  luck 
out  if  wc  are  not  too  narruvly  sighted  and  find  things  that  we  had 
not  oven  been  looking  for.  The  good  and  soak  creative  researcher 
will  even  go  one  atep  .further  and  force  the  process  of  serendipity. 
Outside  of  the  foraal  -vepects  a£  whatever  design  you  adapt  for  a 
particular  wave In a,  chare  ate  side  effects  which  can  bn  looked  at 
to  a  lose  formal  way.  Sloppy?  Yea,  but  also  ctagtlve.  As  wa  stwrted 
out  by  saying,  It's, a  sloppy  world  pl»t  thayo  and  iept  dagrbe  of  slcp- 
plneea  in  dealing  wlth  .lt  may  ha  of  help,  jot t  as  in  soaai  control 
processes,  the  introduction  of  white  nolee  Into  the  ay a  tarn  has  a 
stabilising  effect.  Plan  for  the  unexpected, , then,  cod  the  surprise 
■Ight  well  ba  a  pleasant  ens.  •  ' 
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ii.  nonxti 


In  the  tumr  of  1964,  Rond  mod*  an  Informal  study  of  the  Air 
Oafenae  Command  Dispersal  Plan.  One  policy  contained  In  that  plan 
•toted  that  aircraft  at  the  dispersed  site  mould  be  maintained  on 
ground  alert  status  for  a  maxlmas  of  48  hours;  they  would  then  be 
flown  back  to  their  hosts  bases.  The  requirement  for  a  rotation  period 
of  no  more  than  48  hours  between  dispersed  site  and  horna  base  appears 
to  have  been  based  on  a  widespread  belief  In  ADC  that  aircraft  re* 
liability  deteriorates  rapidly  when  aircraft  are  left  on  the  ground 
for  more  chan  a  few  days.  An  inflexible  requirement  of  this  kind 
appears  to  have  serious  consequences  when  aircraft  ere  at  a  dispersed 
site,  the  consequences  at  the  hosM  base  are  perhaps  less  serious,  but 
are  still  present*  If  a  longer  ground  alert  were  possible  without 
degrading  aircraft  performance,  flight •scheduling  and  maintenance-shop 
manning  would  be  considerably  easier  because  of  greeter  flexibility, 
further,  a  longer  alert  would  lead  to  a  number  of  cost  reductions.  For 
example,  Che  clam  spent  In  uploading  and  downloading  weapons  for  ground 
alert  could  be  cut  to  between  25  and  50  percent  of  the  present  level. 
Since  loading  requires  about  9  to  12  men,  this  saving  alone  could  be 
considerable.  As  a  result  of  these  considerations,  one  of  the  sug¬ 
gestions  made  in  the  Rand  atudy  was  that  the  effect  of  ground-alert 
duration  on  aircraft  reliability  be  determined  experimentally. 

As  a  result  of  the  Rand  recoaBsendation,  Air  Defense  Coemiand  di¬ 
rected  that  a  atudy  be  undertaken.  A  meeting  was  held  at  ADC  head¬ 
quarters  between  various  Air  Force  officers  and  a  Rand  representative 
to  develop  the  experimental  design  snd  procedures.  The  central  Issue 
in  this  first  meeting  concerned  the  tradeoffs  between  a  highly  con¬ 
trolled  and  a  loosely  controlled  design  for  Che  test.  This  issue  Is 
focused  sharply  In  consideration  of  sample  else. 

Estimating  Sample  31«e.  Because  ADC  was  In  a  hurry  to  make  a 
decision  concerning  length  of  ground  alert  it  was  necessary  to  keep 
the  sample  as  small  as  possible.  Ac  the  same  time  the  Type  I  and  II 
errors  both  had  to  be  small.  The  Type  I  error  is  the  probability  of 
deciding  that  differences  in  aircraft  performance  exist  (as  a  result  of 
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difference* /length  of  ground  alert)  when  tn  fact  there  are  no  true 
differences.  The  Type  II  error  It  the  probability  that  true  dif¬ 
ference*  exist  but  are  not  detected,  1.*.,  the  null  hypothec  la  is 
not  rejected.  The  errors  define  the  two  rlaka  taken  by  the  dec It ton - 
maker,  and  both  Involve  certain  costs  to  him.  If  a  Type  1  error  occurs 
he  will  decide  not  to  go  to  longer  alert  periods  (because  differences 
in  aircraft  performance  appear  to  be  related  to  length  of  alert)  and 
lose  the  benefits  he  would  gain  by  using  longer  periods.  If  a  Type  11 
error  occurs,  he  will  decide  In  favor  of  longer  alert  periods  (because 
he  thinks  there  are  no  differences  due  to  length  of  alert)  and  a  de¬ 
grading  of  aircraft  performance  will  result. 

It  le  desired  that  both  of  these  errors  be  samll  and  of  known 
site.  There  were  two  methods  available  for  controlling  the  else  of 
these  errors.  The  first  Is  through  the  use  of  experimental  controls 
which  reduce  random  variation*  (i.a.  the  error  variance,  or  error  of 
measurement  la  smaller).  The  second  Is  through  the  solution  of  an 
adequate  sample  size.  The  calculation  of  sample  else,  while  controlling 
for  Type  I  and  tl  errors  requires  the  following:  (1)  an  estimate  of 
error  variance,  (2)  a  decision  on  the  magnitude  of  difference  (In 
aircraft  performance)  that  la  meaningful  end  Important,  and  (3)  a  stats 
ment  of  the  size  of  risk  the  decisionmaker  is  willing  to  accept,  i.e., 
the  else  of  Type  I  and  II  errors. 

It  was  decided  that  a  change  In  aircraft  break-rate  (proportion 
of  sorties  on  which  aircraft  malfunctioned)  of  0.1  was  important. 

It  waa  also  decided  that  the  Type  1  error  ahould  be  no  larger  than  .05, 
and  the  Type  II  error  ahould  be  approximately  the  aame  alse  for  the 
moat  powerful  testa,  and  could  be  leaa  In  some  caaee. 

Using  the  normal  approximation  for  tests  on  proportions  with 
o  •  . 05  we  have : 


(1) 

’ 

P.  -  P- 

Prob 

1  * 

<  1.96 

\l  2pq/ 

. 
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where  and  fj  k«  the  observed  proportions,  p  ts  tho  eat lasted 
vsluo  of  the  proportion  In  ths  population  and  is 


sod  q  -  1  -  p.  Ths  niabtr  l.9ft  Is  ths  vslus  of  a  normal  variate  (a), 
such  that  Prob  (a  <  1.9ft)  ■  .93,  or  it  is  ths  critical  ration  for  a 
test  whan  o  •  .05.  For  ths  salts  of  as  tint!  at  aaapla  sis*  we  lot 
p  *  0.5.  This  Is  la  the  rsags  of  salons  sapsetad  In  ths  strdy.  Fur ths r- 
mora,  the  product  K  (which  la  used  for  calculations)  changes  vary  little 
over  a  wide  rants  of  p  as  Indicated  In  Table  1. 


Table  1 


Raturnlnf  to  Equation  (1),  and  writing  ths  quantity  within  the 
brackets  as  an  Identity: 


However,  recall  that  P  «  Pj  ■  .1,  and  p  ■  q  -  .5,  wa  hava 


O)  n  -  13ft 

Thus,  If  n  •  13ft.  ths  probability  la  937.  that  p  falls  within  ths 
Intarval  +0.1  of  p. 
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The  power  of  the  toot  (l  *  prop  11  •«•»)  )  for  •  dtfforaeno 

«e  big  aa  0.1  la  of  M«r«0  .SO,  end  for  a  Otfforoooo  m  kt|  M  0.1  It 
U  .175.  ThU  U  *000  U  rtfM  1. 


Critical  Holloa 


Fig.  1  -  DUtrlbutlon  of  -  F2  tor  throe  mime  of  $ 


Far  o  true  difference  (between  the  men  of  dUtrlbutlon  A  end  »)  of 
0.1  the  power  U  ohovs  by  eurre  B,  and  la  reyreeentod  by  the  arm 
under  B  to  right  of  0.1  (the  critical  region).  Obrlouoly  thle  la 

501.  For  e  true  difference  of  0.2,  the  power  le  ebown  by 
cure«  c,  and  egeln  U  repreeentad  by  the  area  to  tha  rl|ht  of  0.1. 

It  la  obwloua  that  thU  la  97. 3X,  elnce  95%  of  tha  dUtrlbutlon  of 
e we.  c  fella  between  0.1  and  0.5  .  ThU  followe  wary  directly  frm 
the  fact  that  the  mane  of  the  dletrlbutlona  of  Fj  •  Fj  (0.0,  0.1 
end  0.3)  are  epaced  at  l.H  etardard  devUtlon  UterveU.  The 
critical  region  apeclfiaa  the  valuer  of  Fj  •  f2  under  *Q  for  «d»leh 
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Hq  it  rejected  (0.1  in  the  example);  conversely,  the  critical  region 
specifies  the  values  of  Pj  *  Pj  for  which  is  accepted.  Power  la 
the  probability  of  accepting  Pl  given  H^i#  true  (i.e.,  K? :  P,  -  Pj  • 

0.1  and  Py  -  Pj  -  0.2). 

Thus  we  conclude  that  (cap lea  of  alae  approximately  136  will 
yield  acceptable  Type  I  and  it  error*.  Ac  will  be  pointed  out  in 
the  following  page*,  the  power  of  a one  teat*  in  the  design  are  better 
than  others,  and  in  some  case*  power  ia  to  be  improved  by  pooling 
procedures . 

General  Deal an.  Ground-alert  periods  of  four  different  lengths 
were  used  in  the  study.  Pour  length*  were  used  In  order  to  determine 
the  shape  of  the  "reliability"  curve  over  time.  Further,  an  even 
number  of  periods  allows  one  to  combine  the  short  vereu*  long  periods 
for  more  powerful  statistical  teste.  This  wee  particularly  desirable 
for  some  indices  of  aircraft  reliability  (e.g.,  ground  aborts)  that 
have  low  frequencies  of  occurrence.  The  periods  of  ground  alart  used 
were  2,  4,  6,  and  8  day*.  A  two-day  period  was  used  because  that 
la  the  length  of  alert  proposed  In  the  ADC  dispersal  plan.  An  eight- 
day  period  was  used  because  it  was  a  consensus  that  aircraft  would 
probably  show  evldance  of  dacreaaad  reliability  after  a  ground -alert 
period  of  this  length.*  As  e  result,  one  could  then  determine  the 
maximum  length  of  ground  alert  poeeible  without  degrading  aircraft 
reliability.  In  order  to  determine  the  generality  of  results,  three 
types  of  ADC  aircraft  (F-101,  P-102,  and  F-106)  and  two  baas*  (rep¬ 
lications)  for  aach  type  were  used  in  the  study. 

Independent  Variable* .  Ssveral  indicators  of  aircraft  reliability 
are  used  in  this  study,  the  primary  one  being  mlselon  capability. 

The  mission-accomplished  rate,  reported  by  the  pilot  end  also  scored 
by  autoeiaeic  devices,  le  the  measure  of  mission  capsbllity.  These 
deta  ware  obtained  from  the  ADC  76-3  Pilot  Debriefing  Forme.  Two 
other  indicators  of  aircraft  rsllsbillty  are  important,  although  not 


This  was  the  opinion  of  some  ADC  and  some  Rend  personnel, 
end  was  thought  to  reflect  the  opinion  of  many  maintenance  personnel 
In  the  field. 
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directly  related  to  mice ion  capability:  pilot -reported  malfunction 
(olao  on  76-3),  and  Air  Fore*  Manual  66-1  maintenance  data.  Experience 
haa  lndleatad  that  tha  number  of  malfunctlona  la  a  vary  aanalttva 
meaaure  and  tha  raaultant  power  of  tha  taata  would  ba  oonaldarably 
higher  than  for  taata  on  break-rate . 

In  addition  to  reflecting  aircraft  reliability,  tha  AIM  66-1  data 
allow  an  eatimatlon  of  Maintenance  coat a  ar  a  function  of  tha  duration 
of  ground  alert.  For  example,  lncreaaed  effort  by  maintenance  crave 
and  tntenalve  ground  teatlng  could  poaalbly  maintain  aircraft  re¬ 
liability  for  extremely  long  per lode  of  ground  alert,  although  the 
coat  for  doing  ao  might  be  high.  fhlle  aircraft  mlealon  capability 
(aa  a  function  of  the  length  of  alert)  la  Che  primary  concern  of  the 
etudy,  the  coat  for  that  capability  cannot  be  diaregarded. 


HfM.AVi  “J.-* :  ‘K':J 


In  thli  •action  tha  experimental  control* **  which  wara  introduced 
to  lacraaaa  tact  sensitivity  ara  stated. 

Aircraft  of  thraa  types  from  six  ADC  haaac  wara  placed  an  ground 
alert  far  parted*  of  1,  A,  A,  and  •  days.  Tha  affect  of  groumd -alart 
dura t lea  oa  aircraft  reliability  waa  aaaaurad  by  aircraft  performance 
on  tha  flrat  aortic  following  tha  ground  alart.  To  raduca  a  owe  of 
tha  random  variation*  In  tha  data,  a award  rulas  and  raatr let Iona 
wara  Ingwaad  on  tast  aortlaa: 

(1)  Tha  aortla  followad  ground  alart  waa  required  ta  taka  place 
within  11  hour*  af  tha  and  af  tha  ground -alert  parted,  and  with  no 
preventive  Maintenance  prior  to  flight. 

(2)  During  die  alart  period,  minor  maintenance  could  ha  per¬ 
formed  aa  long  aa  tha  aircraft  waa  not  removed  from  alart  eeatue. 

(3)  Tha  aortla  flown  Immediately  following  a  teat-alert  perled 
had  to  consist  of  at  least  two  attempted  latereepta. 

(A)  At  least  thraa  sorties  had  to  he  flown  between  consecutive 
teat-alert  periods  for  a  given  aircraft. 

(5)  Whenever  an  aircraft  flaw  a  aortla  following  a  taat-alert 
period,  It  carried  electronic  evaluators  In  tha  weapons  rail*. 

To  control  nonrandom  variation  between  alreraft,  tha  experimental 
design  called  for  every  alreraft  to  ha  placed  on  alart  for  aaeh  of 
tha  four  alart  period*.  (Previous  atudlaa  at  both  Band  and  ADC 
Lndleate  that  significant  differences  exist  hatwaan  alreraft  for  short 
periods  of  tle»e  (aaverel  months).  Tha  order  In  which  a  given  air¬ 
craft  waa  placed  in  each  of  tha  alart  periods  night  possibly  have  an 
affect,  l.a.,  tha  alart  period  sequence  of  2-4-6-0-daya  night  have  an 
efface  different  from  tha  sequence  of  8-4-6-2-days.  This  kind  of 
order  affect  could  be  controlled  by  counterbalancing  ao  that  all 

*W.  H.  McCiothlln  and  T.  8.  Donaldson,  Trends  In  Aircraft 
Maintenance  Requirement*  (For  Official  Lae  Only),  Tha  Rand  Corporation, 
RM-4049-HL,  (DDC  No,  AD  447-880),  June  1964;  T,  S.  Donaldson  and 
Anders  Sweet  land,  Trends  In  F-101  Maintenance  Requirements,  The  Rsnd 
Corporation,  RM-4930-fU,  April  1966  (For  Official  Use  Only), 

**Unpubllshad  study  by  Captain  John  B.  Abell,  Headquarters  Air 
Defense  Coaamnd  (ADMLP). 
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sequences  occur  with  equal  frequency.  To  hew*  conducted  the  expert* 
wont  with  count erba lancing,  however,  would  hew*  Impo  .were  restric¬ 
tions  on  aircraft  scheduling.  Since  th*a*  order  effects  were  expected 
to  be  smell,  they  were  controlled  by  randomisation.  This  order  ran¬ 
domisation  wee  eerrled  out  by  ADC  Headquarters . 

Since  there  were  ewe  bee**  for  each  elrcreft,  the  deeitn  might 
be  thought  of  ee  a  three -way  leyont  Involving  thro*  factor*!  alert 
period,  aircraft  type  and  replication*  (beaea).  The  experimental 
design  Is  ehewn  In  Table  2.  The  design  euggeste  *1x3x2  enelyels 
of  v«r lance,  with  the  wlthin-eell  error  verlcncc  partitioned  Into 
a  betwaen-alrcraft  and  realdual  error.*  Difference*  between  aircraft 
type  end  between  replication*  would  be  teated  against  the  between* 
aircraft  error,  end  difference*  between  alert  period*  would  be  teated 
against  the  realdual  error.  This  type  of  analysis  with  the  kind  of 
data  obe erred  in  the  study  (primarily  dlohotomoue)  and  with  con¬ 
servative  sample  sites  would  be  extremsly  risky  since  the  true  Type  I 
end  It  errors  would  not  bo  known.  This  la  particularly  true  with 
respect  to  the  interpretation  of  Interaction  of foots,  which ,  If  present, 
would  then  require  an  enelyels  of  on*  effect  et  each  level  of  one  of 
the  ether  effects  ("simple  ef facte"  analysis).  Rather  than  follow 
this  "classical"  analysis,  the  above  comments  suggest  that  differences 
between  alert  periods  should  be  analysed  for  each  base  separately, 
Pooling  across  bases  would  follow,  «dion*var  possible,  to  give  more 
powerful  tests.  This  Is  the  analytic  procedure  used  In  this  study. 

Por  example,  •#*  B.J.  Winder,  Statistical  Principles  In 
Experimental  Dealen.  McGraw-Hill  Book  Coa^any,  Inc. ,  New  York, 

1962,  Chap.  7 


Table  2 


Aircraft  reliability  was  assessed  by  the  following  Indices: 

(1)  Break  rate  by  category:  An  aircraft  la  placed  In  one  of 
three  categories  after  a  sortie. *  These  categorlaa  and  codas  era: 

Code  1.  The  aircraft  la  operationally  ready*  i.e.,  all 
Installed  systems  are  fully  operational . 

Code  2.  The  capability  of  the  aircraft  to  perform  on 
air  defense  mission  Is  degraded. 

Code  3.  The  aircraft  Is  not  capable  of  performing  an  air 
defense  mission. 

The  proportion  of  Code  2  or  Code  3  sorties  relative  to  the  total 
number  of  sorties  flown  (or  sum  of  Codes  1,  2,  and  3)  is  the  break 
rate  fot  that  respective  code. 

(2)  MA  success  rate:  HA  denotes  mission  accomplished;  it  re¬ 
quires  that  a  fire  algnal  must  register  on  the  pilot's  display  scope 
and  the  target  dot  be  In  the  center  of  the  radar  screen. 

*ADC  Regulation  66-28,  7  October  1964,  p.7. 
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(1)  WSEM/MSR  success  rate;  This  refers  to  Che  aueeeaa  of  Che 
aircraft  systems  In  perfortalng  a  mission  aa  meaaured  by  Che  WSEM  aid 
MR  electronic  evaluators. 

(4)  Humber  of  aborta  on  around  alert:  A  ground  abort  refere  to 
a  cancelled  aortic  due  to  e  crttlcel  failure  occurring  before  takeoff. 
(Weather  and  operations  ground  aborta  were  not  coneldered). 

(3)  Number  of  ln»fllaht  erne  nenc  lea:  An  in-flight  emergency 
oecura  whan  the  mission  auat  be  cancelled  aa  the  reault  of  a  critical 
failure  while  In  flight. 

(6)  Number  of  dlacreoanclaa  oar  sort  la:  This  refera  to  the 
total  amber  of  malfunctions  reported  by  the  aircrew. 

(7)  Maintenance  on  the  aircraft:  Ihle  la  defined  ea  the  un- 
acheduled  flight-line  maintenance  ea  recorded  on  the  AFM  66-1,  AFTO  210 
and  211  forms.  Units  produced  and  maintenance  manhours  were  used  to 
measure  the  amount  of  maintenance  required  by  the  aircraft. 

the  first  six  of  these  measures  are  taken  from  Information  on 
ADC  Form  76-3  (pilot  debriefing).  This  form  la  made  out  after  every 
sortie;  it  was  modified  for  the  present  study  to  Include  a  code  in¬ 
dicating  which  test  alert  period  the  wortle  followed.  The  seventh 
measure  was  obtalnad  from  AFM  66-1  forms.  Unfortunately,  these  forms 
were  not  identified  by  sortie,  making  the  resultant  analysis  weaker 
than  it  might  otherwise  have  bean. 
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1Y,  .  RES  ULTS 


Sorties  »r»  attempted  following  440  ground  olorea.  The  number 
of  alerts  for  each  boat  and  for  each  alorc  period  ara  shown  In  Tabla  3. 
The  experimental  daalgn  required  that  ovary  aircraft  racalva  all  treat¬ 
ment*  (langth  of  ground  alart),  and  that  tha  ordar  In  which  an  aircraft 
was  exposed  to  tha  treatments  ba  ratdaa.  Inspection  of  tha  actual  dates 
that  alroraft  in  tha  study  ended  their  alert  periods  indicated  that  no 
biaaas  in  tha  ordar  of  treatamnt  presentation  were  present,  and  experi¬ 
mental  differences  due  to  order  effects  should  be  absent  (or  at  least 
extremely  small).  From  this,  we  conclude  that  the  bases  were  able  to 
follow  Che  scheduling  requirements,  and  that  in  general,  the  experimental 
design  was  followed. 


Tabla  3 

DISTRIBUTION  OF  SAMPLE  FOR  EASES 
AND  TREATMENTS 


Alert  Period  (days) 

Base 

2 

4 

6 

8 

Total 

1 

13 

17 

14 

14 

60 

2 

16 

16 

16 

15 

63 

3 

22 

22 

22 

22 

88 

4 

20 

18 

20 

17 

75 

5 

21 

21 

22 

22 

86 

6 

17 

17 

17 

17 

68 

Total 

111 

111 

111 

107 

440 

It  was  obvious  by  inspection  of  the  means  for  each  base  that 
differences  of  a  meaningful  sort  did  not  exist  between  alert  periods. 

The  actual  data  are  presented  in  the  study  by  Donaldson  and  Burke 
(1966),  but  are  omitted  in  this  peper  because  of  their  security  clssslfi- 
catlon.  The  data  however  did  not  reveal  any  Increase  in  any  of  the 
indlcatora  aa  a  function  of  the  length  of  ground  alert,  and  in  fact, 
more  often  than  not  the  longer  alerts  indicated  better  aircraft  per¬ 
formance.  None  of  the  differences  however  were  significant  when  tested 
with  x2. 


The  date  across  bases  vara  pooled  In  order  to  increase  the  power 
.  ..  Qf  (he  testa,  but  again  no  significant  differences  were  found. 

A  compilation  arose  in  the  analysis  when  It  was  discovered  that 
some  aircraft,  ware  sheltered  and  some  were  not.  The  two  F-106  bases 
sheltered  all  aircraft,  and  these  praaented  no  problem:  However,  the 
other  four  braes  sheltered  some  of  their  aircraft,  but  not  all.  An 
analysis  ct  sheltered  vo .  housheltered  aircraft  showed  a  highly  sig¬ 
nificant  difference  in  all  hut  one  of  the  76-3  measures  in  fsvor  of 
the  sheltered  aircraft,  The  one  measure  which  did  not  show  this 
effect  was  the  WSEH/MSR ,  although  it  did  show  that  sheltered  aircraft 
on  long  alerts  were  significantly  better  than  those  on  shorter  alerts. 
This  was  thought  possibly  to  bo  due  to  drying  out  in  the  shelters. 

Analysis  of  tha  66-1 .unscheduled  maintenance  man-hour  data  In¬ 
dicated  that  during  long  alerts,  aircraft  re<tuire  only  slightly  more 
maintenance  chan  during  short  ones.  Hie  coat  of  chis  Increase  1b 
negligible.  Further,  there  wea  no  indication  of  a  differential  cost 
as  a  function  of  slsrc  duration  for  a  period  of  up  to  three  days 
following  the  aler:. 

The  results  of  the  study  clearly  Indicated  that  aircraft  left 
on  alert  for  eight  days  were  as  mission  capable  aa  those  left  on 
alert  for  two  days. 


■  *<*»  4l<tsM«u.444.  •  &>* 


V.  CONCLUSIONS 


This  study  Is  an  example  of  how  experimental  controls  can  be 
used  in  an  operating  environment.  A  field  test  of  this  sort  re* 
quires  that  the  Initial  problem  be  well  defined,  and  that  the  design 
of  the  test  is  clearly  and  simply  related  to  the  problem.  Too  often 
test  designs  become  unwieldlv  and  overly  cocplex  because  the  design 
Incorporates  too  many  operations.  A  field  test  of  this  kind  is  de¬ 
signed  to  obtain  relatively  precise  information  about  a  specific 
problem  and  a  few  variables  rather  than  uncertain  and  confounded 
information  about  a  wide  ranging  problem. 

In  all  respects  this  study  was  considered  to  be  a  success,  and 
ADC  policy  was  changed  within  a  few  months  of  completion  of  the 
sCudy . 

What  we  have  been  talking  about  so  far  may  be  thought  to  fall 
under  the  general  heading  of  the  classical  approach  to  experimental 
design.  This  approach  derivea  mainly  out  of  the  thinking  of  R.  A. 

Flaher  during  the  early  part  of  this  century,  end  even  until  the 
present  day  It  dorlnates  the  teaching  In  the  areas  not  only  of  experi¬ 
mental  design  but  also  of  statistics.  Other  approaches  ere  beginning 
to  emerge,  however,  and  we  should  be  aware  of  possibilities  for  different 
experimental  strategies  which  are  available  to  us  because  of  these  new 
ways  of  thinking.  We  will  just  mention  some  of  these  briefly  in  con¬ 
clusion,  without  giving  any  specific  examples. 

The  first  of  the  two  approaches  which  we  went  to  mention  we  will 
call  the  Decision-Theoretic  or  Bayesian  approach.  We  do  experiments 
or  field  tests  because  we  need  to  gather  evidence  for  some  decision 
we  have  to  make,  but  we  generally  do  not,  as  the  classical  theory  would 
have  us  believe,  go  Into  the  field  or  start  an  experiment  In  a  state  of 
total  ignorance.  We  do  have  some  prior  opinions  about  what  is  going 
to  happen  and  we  want  to  collect  data  so  we  can  upgrade  those  opinions. 
That  is  what  the  Bayeslans  are  all  about  -  modifying  prior  opinions 
(generally  stated  In  terms  of  probabilities)  by  means  of  new  evidence. 
They  get  Che  name  Bayeslans  because  the  formally  correct  way  of  adjusting 
prior  probabilities  is  by  applying  Bayes'  Theorem,  but  a  more  distinctive 
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attribute  of  this  group  Is  the  uee  they  make  of  whet  hes  coete  to  be 
celled  subjective  or  per a one 1  probability.  The  personal  probabilities 
ere  concerned  with  the  la^ect  of  eech  Item  of  deta  that  Is  gathered 
on  the  prior  opinions  about  the  hypotheses ,  end  there  la  concern  among 
this  group  with  developing  optimal  stopping  procedures  for  the  collecting 
of  Information.  In  the  decision  theoretic  sense,  there  is  no  point  in 
collecting  any  more  data  when  the  cost  of  information  exceeds  the  ex- 
pected  gain  it  reflects  in  the  payoff.  There  is  an  analogy  here  to  the 
optimal  stopping  techniques  used  In  Wald's  sequential  testing  procedures, 
except  in  the  Bayesian  analysis  there  is  no  concern  with  the  null- 
hypothesis  testing.  The  bayesians,  in  fact,  consider  the  null-hypo¬ 
thesis  concept  to  oe  an  artificial  constraint  which  acta  much  to  the 
detriment  of  most  experimental  design  efforts. 

Another,  even  further  out,  technology  seems ‘to  be  emerging  in 

e 

areas  which  are  concerned  primarily  with  the  evaluation  of  programs  in 
such  fields  as  education,  welfare,  and  social  reforms.  There  is  as  yet 
no  collective  name  for  the  procedures  which  are  being  de'-rloptd ,  but 
we  can  refer  to  them  under  the  designation  of  Environmental  ov  Situational 
Theory.  The  distinctive  feature  of  this  activity  is  that  the  evaluation 
is  generally  done  in  the  absence  of  anything  we  might  call  a  true  control 
group.  Educational  changes,  for  example,  are  so  all  pervading  that  the 
variables  Involved  in  studying  new  classroom  techniques  must  be  con¬ 
cerned  not  only  with  the  techniques  themselves,  but  aleo  with  the  group 
selected  for  study,  the  teachers  who  administer  the  program,  and  the 
evaluation  team  which  exist  gather  performance  messurea  for  the  program. 
The  beat  we  can  do  in  cases  like  this  is  make  an  estimate  of  how  the 
group  would  have  performed  under  the  old  methods  and  compare  this  es¬ 
timate  against  the  results  obtained  from  the  Innovations. 

Marcia  Cuttentag  of  the  City  University  of  New  York,  in  an  as 
yet  unpublished  paper,  advocates  a  legal  model  for  the  evaluation  of 
research  done  in  the  field  in  areas  where  the  institution  of  tight 
controls  is  eil  but  Impossible.  This  model  would  take  as  ita  method 
of  ascertaining  the  "truth,"  the  way  In  which  our  legal  system  operates. 
There  are  rules  which  govern  the  presentation  and  evaluation  of  evidence, 
individual  cases  are  argued  in  terms  of  whether  they  do  or  do  not  fit  a 
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particular  precedent,  with  a  citation  of  the  characteristics  of  the 
case  which  lead*  to  the  Inference  that  It  falls  under  the  precedent. 
Counter  arguments  are  offered  by  the  opposing  side  to  show  why  sueh 
Inferences  are  not  tenable  and  other  plausible  alternative  interpreta¬ 
tions  of  the  data  are  offered.  Evidence  is  classified  according  to 
whether  it  Is  direct  or  circumstantial  and  asaertlons  of  fact  are 
separated  from  direct  evidence.  The  jury,  or  in  this  cese  the  experl- 
mentor  or  the  evaluation  team,  comes  to  a  finding  based  on  the  "pre¬ 
ponderance  of  evidence."  We  can  look  at  this  procedure  as  a  new  form 
of  an  experlgient  in  which  the  experimental  variables  might  be  con¬ 
sidered  a  form  of  advocacy  for  a  certain  position  while  the  controls 
can  be  looked  at  aa  advocacy  for  ocher  position*.  This  la  all  to 
a.<y  that  there  may  be  some  quite  reasonable  alternatives  to  the 
classical  experimental  method  for  drawing  lnfarancee  from  controlled 
observation*  and  theae  methods  do  not  necessarily  leave  us  in  a  mire 
of  subjective  chaoa. 

Let  us  consider  on*  final  example  of  an  axparimental  or  aclantlflc 
method  which,  while  far  from  the  classical  paradigm,  still  has  a  logic 
and  rigor  about  it  lending  it  sign!. leant  respectability.  The  field 
is  anthropology  or  archaeology.  The  scientist  starts  out  in  the  field 
with  only  s  vague  notion  of  what  ha  is  looking  for.  When  he  finds 
something,  he  formulate*  hypotheses  about  the  totality  of  what  la  there 
which  give*  him  not  only  a  picture  of  the  history  he  is  uncovering  but 
also  some  direction  of  where  and  what  to  look  for  as  he  continues. 

Hie  next  finding  may  reinforce  or  negate  his  previous  hypotheses,  and 
he  continues  in  this  cycle  of  looking,  finding,  and  formulating  until 
a  consistent  picture  emerges.  An  experiment  in  the  classical  sense  is 
never  formulated,  but  clear  and  unambiguous  results  often  emerge  from 
the  process. 

There  is  some  indication  from  other  disciplines,  therefore,  that 
something  la  to  be  gained  by  abandoning  a  rigid  adherence  to  classical 
experimental  methodology.  The  example  we  presented  earlier  represented 
an  extension,  a  stretching,  of  the  classical  methods  to  meet  the  con¬ 
tingencies  of  experimentation  under  field  conditions.  With  the  world 
becoming  more  complex,  it  is  incumbent  upon  us  to  expand  our  Imaginations 
even  further  to  meet  the  demands  we  will  have  to  face. 
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A  STATISTICAL  ANALYSES  OF  DYNAMIC  RESPIRATORY  DATA 
GENERATED  THROUGH  PROTECTIVE  MASK  HEAR 

Edward ■  N.  Flak* 

Systems  Analysis  Office 
Eagawood  Arsenal,  Maryland 

I,  INTRODUCTION.  For  years  Defense  Development  and  Engineering  Lahore- 
tor las  at  Edgewood  Arsenal  desired  a  means  of  eaaauring  the  dynamic  per¬ 
formance  of  a  protective  mask,  specifically,  breathing  realetance  under 
field  conditions.  A  system,  designed  end  built  by  Sahborn-Statham,  has 
been  used  with  reasonable  success  in  the  laboratory}  however,  this  system 
is  too  heavy  end  bulky  for  field  use.  After  several  designs  had  been 
submitted  for  acceptance,  Bdgevood  Arsenal  decided  to  develop  a  system 
called  Dynapsr,  manufactured  by  Space  Laboratories,  Inc.,  Van  Itays,  Cali¬ 
fornia.  The  system  consists  of  the  following  unltsi  (1)  a  backpack  having 
a  transmitter  and  a  rechargeable  battery;  and  (2)  a  ground  receiving  con¬ 
sole. 

Mian  Edgewood  Arsenal  Physical  Protection  Laboratory  received  the 
Dynaper  eyatsm,  a  aeries  of  teats  were  conducted  to  determine  its  reproduc- 
tlblllty  over  repeated  runs  and  to  determine  its  accuracy  for  measuring 
mask  resistance  aa  compared  to  that  of  the  Sanborn  laboratory  system. 

The  results  of  these  teats  showed  that  there  la  a  95  percent  chance 
that  the  Dynaper  system  will  measure  a  given  pressure  drop  or  flow  rate 
within  1  percent  for  repeated  rune.  The  difference  between  the  Dynaper  and 
the  Sanborn  system  over  the  flow  range  of  -250  to  +250  liters  par  minute 
averages  approximately  3  percent.  The  results  revealed  that  the  Dynaper 
ays t am  is  an  acceptable  system  for  field  use. 

Since  the  preliminary  tests  on  the  Dynaper  system  were  considered  a 
success,  the  Physical  Protection  Laboratory  initiated  a  test  program  to 
measure  the  mask  resistance  of  three  protective  masks  under  both  field  and 
laboratory  conditions.  The  three  masks  tasted  are  designated  as  the  M17, 

XM28,  and  the  M17A1.  The  test  program  was  statistically  designed  by 
the  author. 

II.  METHODOLOGY.  A  5x5  factorial  design  was  chosen  for  the  teat  program. 

The  two  factors  were  flow  rate  and  breathing  systems.  Flow  rates  of  50,  100, 
150,  200,  and  250  liters  per  minute  were  chosen.  The  following  systems  were 
utilized!  (1)  a  reciprocating  piston-type  engine  celled  a  breathing  machine 
attached  to  the  Sanborn-Statham  recorder;  (2)  the  breathing  machine 
attached  to  the  Dynaper  backpack;  (3)  a  human  subject  on  an  ergometer  carrying 
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a  Dynaper  baekpact  trirad  to  tha  ground  racalvlng  console  In  nha  laboratory; 

(A)  a  huaan  aubjact  exposed  to  fiald  conditions  carrying  a  Dynapar  backpack 
containing  a  radio  transmitter  that  transmits  impulses  to  tha  ground  recei¬ 
ving  conaoia  in  tha  laboratory;  add  <S)  a  huaan  aubjact  on  a  argoaatar 
attached  to  the  Sanborn-Stathaa  recorder.  Tha  first  four  devices  era  con* 
aldered  tha  four  basic  ays teas  that  vara  coaperad  to  tha  standard,  tha  fifth 
systaa,  for  tha  statistical  analyses.  Three  separata  experiments  vara  con- 
ducted  consisting  of  30  dynaalc  runs  covering  a  contiguous  flow  range  of 
-250  to  +250  liters  par  ainuta  vara  performed  and  tha  flow  rataa  and  prassura 
drops  vara  recorded  on  scrip  charts.  Tha  data  on  tha  strip  chart  vara  re¬ 
duced  to  peak  prassura  drops  at  tha  five  flow  rates  chosen.  Since  tha 
early  tests  indicated  that  variation  batvean  runs  was  not  significantly 
different,  data  extracted  from  the  strip  charts  vara  averaged  over  runs  (Table 
1  and  2). 

III.  ANALYSIS.  Analyses  of  variance  (Table  3  and  4)  carried  out  on  tha  data 
reveal*3  that  there  Is  a  significant  variation  between  tha  five  measuring 
systems  for  each  of  the  three  masks,  and  inhalation  and  exhalation  respec¬ 
tively.  Therefore,  regression  analyses  vers  performed  on  the  data  to  deter¬ 
mine  the  functional  relationship  betveen  the  standard  and  tha  four  basic 
syatems.  Tha  regression  analyses  revealed  that  there  la  a  linear  function, 

Y  •  MX,  betveen  the  pressure  drop  value  for  each  of  tha  basic  systems  (X) 
and  the  preesure  drop  value  for  the  standard  ayatea  (Y).  The  additional 
variation,  that  would  be  explained  by  a  higher  order  fit  over  the  linear  fit 
is  not  significant.  The  M  values  for  tha  model  Y  -  MX  are  listed  in  Table  S. 
These  M  values  are  all  significantly  different  from  one. 

Additional  regression  analyses  vara  carried  out  to  determine  the 
functional  relationship,  Y  -  MX  batmen  tha  tvo  breathing  machine  (X)  systems 
(Dynaper  and  Sanborn)  and  their  respective  human  subject  systems  (Y)  for  each 
of  the  three  masks.  These  regression  r-Wysea  revealed  that  the  curves  for 
both  comparisons  vare  not  significantly  •  fferent.  Hence  their  respective 
M  values  vare  pooled  for  each  of  the  thioe  masks.  These  values  vers  also 
found  to  be  significantly  diffarent  from  one.  These  values  are  listed  In 
Table  6. 

IV.  DISCUSSION.  The  analyses  revealed  that  the  Dynaper  field  system,  on 
the  average,  measures  values  1  percent  higher  than  the  Dynaper  laboratory 
system  would  measure  but  the  overall  difference  betveen  values  measured 
by  the  Dynaper  leboratory  system  and  the  Sanborn  system  Is  2  percent. 

When  the  data  ware  plotted  the  mask  resistance  curves  Indicated  that  tha 
relationship  ba tween  the  pressure  drop  end  flow  rate  might  have  a  second  order 
fit  for  all  masks  and  measuring  ays  toms;  but  tha  analyses  indicate  tha  curvet 
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were  linear,  alnea  the  coefficient  of  deter* inetlon  (r  )  for  pressure  drop 
and  flow  rate,  which  rapraaanta  the  total  amount  of  variation  axplelnad  by 
a  linear  curve,  la  greater  than  98  percent. 

Table  S  can  be  uaed  tr  evaluate  the  performance  of  other  protective 
masks  with  the  aame  charec'.erietlc?  aa  either  the  XM28,  HI 7X1,  or  HI 7 
masks.  If  one  dealgna  a  mask  *rith  'he  eaae  character  let  ice  aa  the  XM28 
aaak,  one  would  uae  the  figurea  Hated  under  Dynaper  Human  subject  (labor* 
atory)  ayatam  to  determine  what  the  standard  would  be.  Tor  example.  If  the 
Dynaper  system  measures  a  pressure  drop  of  90sm  of  water  across  ths  mask 
(inhalation)  the  standard  pressure  drop  would  be  87.3  {(0.97)  x  (90)]  am 
of  vatar. 

Table  6  can  be  uaed  to  convert  a  breathing  system  reading  to  its 
corresponding  human  subject  system  reading.  Tor  example,  if  a  mask  similar 
to  the  Ml 7  is  fitted  to  the  mechanical  breathing  head  aat  to  produce  a  flow 
rate  of  160  liters  per  minute,  and  the  Dynaper  system  measures  the  pressure 
drop  acroaa  the  mask  to  be  80m  of  water,  the  same  Dynaper  backpack  if 
carried  by  a  human  subject  breathing  at  160  liters  par  mlnutea  through  the 
particular  mask,  would  measura  a  pressure  drop  of  96  [(120)  x  (60)]  ami  of 
water. 

VI.  CONCLUSION.  It  can  be  concluded  from  Che  test  program  thati  (1)  The 
Dynaper  system  la  a  reliable  means  of  measuring  ths  dynamic  performance  of 
any  protective  mask  exposed  to  field  conditions;  (2)  the  system  cen  measure 
the  mask  resistance  of  a  mask  under  field  conditions  within  3  percent  of  ths 
mask  resistance  determined  in  the  laboratory  with  the  standard  equipment; 

(3)  the  system  has  a  93  percent  chance  of  measuring  a  given  mask  resistance 
within  1  percent  for  repeated  rune;  (4)  the  mask  resistance  curves  for 
each  mask  are  linear;  (5)  ths  mask  resistance  for  the  XM28  la  significantly 
higher  than  for  either  the  M17A1  or  M17  mask;  and  (6)  the  factors  established 
to  correlate  field  data  with  data  generated  in  tha  laboratory  can  be  used  to 
evaluate  the  performance  of  other  protective  masks  with  tha  earns  characteristics 
aa  the  three  meake  tested 


59 


Table  1 


KEAN  PRESSURE  DROP  AT  SEVERAL  PLOW  RATES,  HL7,  XH2«,  H17A1 
PROTECT MASKS,  FIVE  MEASURING  SYSTEMS -INHALATION 


Flow  Rates  (Llter»/Mln)  |  50  I  '  100  t  150  1  200  1  ~250’ 


Hack 


o!  Water 


Mcesuring  Sy atoms 


Ml? 

Senborn-Brea thing  Machine 
Dynaper-Brea thing  Machine 
Dynaper-Human  Subject  (Laboratory) 
Dynaper-Human  Subject  (Field) 
Sanborn-Huraan  Subject 

22 

24 

30 

27 

30 

1 

n 

103 

112 

138 

124 

130 

1 

Sanborn-Breathing  Machine 

25 

55 

88 

in 

165 

Dynaper-Brea thing  Machine 

29 

60 

96 

m 

175 

XM28 

Dynaper-Human  Subject  (Laboratory) 

30 

65 

105 

145 

190 

Dynaper-Human  Subject  (Field) 

33 

63 

103 

143 

187 

Senboro-Human  Subject 

25 

56 

93 

134 

180 

Senborn-Breathlng  Machine 

m 

50 

81 

■K  [3 

m 

Dynaper-Breathlng  Machine 

58 

86 

SjgSff  J 

m 

M17A1 

Dynaper-Human  Subject  (Laboratory) 

61 

96 

fliC  i  J 

175 

Dynaper-Human  Subject  (Field) 

mi 

65 

106 

149 

163 

Sanborn-Human  Subject 

31 

65 

101 

145 

190 

Table  2 


MEAN  PRESSURE  DROP  AT  SEVERAL  FLOW  RATES,  M17,  XM28,  M17A1 
PROTECTIVE  MASKS,  FIVE  MEASURING  SYSTEMS -EXHALATION 


B1±M 

vmm 

EBR21 

Measuring  System? 

mm  of  Water 

Sanborn-Breathing  Machine 

-10 

-22 

-30 

-40 

-48 

Dynaper-Breathlng  Machine 

-15 

-27 

-37 

-44 

-50 

P.17 

Dynaper-Human  Subject  (Laboratory) 

-10 

-21 

-30 

-37 

-43 

Dynaper-Human  Subject  (Field) 

-12 

-23 

-31 

-40 

•45 

Sanborn-Human  Subject 

-  9 

-20 

-28 

-35 

-40 

Sanborn-Breathing  Machine 

-10 

-20 

-30 

-40 

-50 

Dynaper-Breathlng* Machine 

-11 

-24 

-33 

-43 

-53 

XM28 

Dynaper-Human  Subject  (Laboratory) 

-10 

-19 

-28 

-36 

•46 

Dynaper-Human  Subject  (Field) 

-11 

-21 

-31 

-41 

-51 

Sanborn-Human  Subject 

-12 

•22 

-31 

-40 

-49 

Sanborn-Breathing  Machine 

-10 

-23 

■ESI 

-60 

-85 

Dynaper-Breathlng  Machine 

-13 

-30 

B3 

-69 

-93 

M17A1 

Dynaper-Human  Subject  (Laboratory) 

-12 

-25 

K] 

-58 

-76 

Dynaper-Human  Subject  (Field) 

-11 

-24 

E] 

-55 

-73 

Sanborn-Human  Subject 

-  9 

-23 

m 

-54 

•68 

NTfl 


Table  3 


ANALYSES  OP  VARIANCE  OF  DATA  IN  TABLE  1 
_  M17  Mask 


Between  Flows 
Between  Systems 
Error 

Total 


56528.0 

1918.0 

A10.0 

58856.0 


14132.0 

479.5 

25.6 


552.9 

18,7 


Source  of  Variation 

<1P 

SS 

.  _KS_ 

F  Ratio* 

between  Flows 

4 

71903*6 

17975.9 

166.4 

Between  Systems 

4 

798.8 

199.7 

18.5 

Error 

16 

173.6 

10.8 

- 

M17A1 


Between  Flows 
Between  Systems 
Krror 

Total 


67274.8 

2050.4 

512.8 


16818.7 

512.6 

31.4 


535.6 

16.3 


Table  4 

ANALYSES  OF  VARIANCE  OF  DATA  IN  TABLE  2 


Source  of  Variation 

_ dE _ 

SS . 

MS 

Between  Flows 

Between  Systems 

Error 

4 

4 

16 

3626.2 

186.6 

21.8 

906.6 

46.7 

1.4 

Total 

24 

3834.6  {  I 

647.6 

33.3 


Between  Flows 
Between  Systems 


4686.’ * 
65.8 


1171.7 

16.4 


1065.2 

14.9 


H17A1 


8ource  of  Variation 

d? 

SS 

MS 

Between  Flows 

16 

14556.4 

3639.1 

Between  Systems 

16 

446.8 

111.7 

Error 

4 

212.8 

13.3 

24 

15216.0 

♦All  F  Ratios  significant  at  957.  level, 


273.6 

8.4-- 
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Table  5 


M  VAIfUSS  FOR  RELATING  PRESSURE  DROPS  MEASURED  BY  FOUR  BAXIC 
MEASURING  SYSTEMS  (X)  TO  THOSE  MEASURED  BY  THE  STANDARD 
SYSTEM  00  THROUGH  THE  FUNCTIONAL  RELATIONSHIP,  Y  -  MX 


Breeching 

Method 

Mask 

Type 

Sanborn 

Breathing 

Machine 

Dyneper 

Breathing 

Machine 

Dyneper* 

Human 

Subject 

(Laboratory) 

Dyneper* 

Hunan 

Subject 

(Field) 

Ml  7 

1.19 

1.16 

0.95 

1.05 

Inhalation 

XM28 

1.11 

1.06 

0.97 

1.00 

M17AI 

1.22 

1.18 

1.09 

0.99 

WSMM 

0.82 

0.87 

0.94 

0.93 

Exhalation 

0.92 

0.89 

1.03 

0.92 

B 

0.80 

0.75 

0.92 

0.95 

Table  6 

M  VALUES  FOR  RELATING  PRESSURE  DROPS  MEASURED  BY  THE 
BREATHING  MACHINE  (X)  TO  THOSE  MEASURED  BY  THE  HUMAN 
SUBJECT  SYSTEM  00  THROUGH  THE  FUNCTIONAL  RELATIONSHIP 

(Y  -  MX) 


Breathing 

Method 

Protective  Maska 

Ml  7 

XM28 

M17A1 

Inhalation 

1.20 

1.10 

1.14 

Exhalation 

0.87 

0.89 

0.81 
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A  COMPLEX  SPLIT  PLOT  DI8XOtl  POO  AM  EXPERIMENT 
INVOLVING  STAMO  SMALL  UNIT  LIVE  PIKE 

Jim  8.  Da  OrioU,  Q.  David  halktabirry,  Tima  K,  Rodgers 
SPS  Harvey  Bunce,  lit,  U.S.  Any,  CDCEC 
Litton  Scientific  Support  Laboratory 
Mellonles  Systems  Development  Division 
Litton  Iaduatrlaa 
Port  Ord,  California 

1.  GENERAL  DISCUSSION  OF  EXPERIMENT. 

a.  Objective  of  the  Experiment.  Tha  objective,  with  raapact  to  parfor- 
mance,  of  the  STANO  Small  Unit  Llva  Flra  Experiment  waa  to  obtain  flra  effec- 
tivanasa  data  of  a  aaall  unit  firing  night  llva  flra  with  aalactad  STAMO 
davlcaa  undar  varying  lavala  of  illumination,  varying  tarraln,  employing 
various  typos  of  anunltion  (including  tracar)  against  rsprasantatlva  porsonnol 
targots.  This  objsctlvs  waa  than  paraphrased  Into  a  sat  las  of  qua st Iocs, 
toned  Sssantlal  Elements  of  Analysis  (ESA),  which  could  ba  answered  analyti¬ 
cally.  The  Min  EEA  for  this  experiment  vast  Does  a  tan  aan  flfla  squad 
reinforced  with  a  3  aan  aachlnagun  section  have  a  significant  increase  in  fire 
effectiveness  when  using  one  test  mix  of  STAMO  devices  as  coopered  to  its  fire 
effectiveness  when  using  another  test  alx,  or  when  using  no  night  vision  aldsT 

(1)  In  natural  low  light? 

(2)  In  natural  aid  light? 

(3)  In  natural  high  light? 

To  answer  this  BEA  four  variables  ware  selected  to  ba  analysed.  These  varia¬ 
bles,  termed  Measures  of  Effectiveness  (HOE),  were: 

(1)  MOE  1:  THTP  -  Target  Hit  per  Targets  Presented.  The  proportion  of 
targets  presentod  that  we7a  hit  nsasures~*the  target  effects  achieved  and  there¬ 
fore  is  the  basic  measure  of  fire  effectiveness. 

(2)  MOE  2:  TFE  -  Time  to  First  £ffeet.  The  tine  from  the  appearance  of 
che  first  target  in  any  array  to""the  registration  of  tha  first  hit  of  near 
ales  on  any  target  in  that  array.  This  MOB  measures  the  acquisition  time  plus 
the  time  required  to  achieve  a  target  effect  and  thus  will  provide  an  indica¬ 
tion  of  the  relative  target  acquisition  tine. 

(3)  MOE  3:  FRPET  -  Fractional  Reduction  of  Programmed  Exposure  Time. 
Reduction  of  Programmed  Exposure  Tlme7AdJusted  (for  malfunctions)  Programmed 
Exposure  Tlae  (RPBT/APET).  The  proportional  reduction  of  programmed  exposure 
time  due  to  target  hita  measures  the  timeliness  of  tha  effective  flra  of  the 
squad  end  reflects  the  number  of  targets  that  vara  hit  as  well  as  tha  time  to 
hit. 
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(4)  HOK  4i  THSA.B  -  Targat  Hit*  p*r  round  of  Ammunition  Expanded.  Th* 
•V«r*s*  single  round  hit  probability  ia  a  basic  meaTure  of  th*—aqu*d 1  a  efficiency 
in  achieving  target  affacta. 

b.  Field  Experiment.  Tha  fiald  experiment  was  conducted  at  Huntar  Llggat 
Military  R* nervation.  Two  targat  arrays  vara  utllisad  during  each  trial. 

A  total  of  thraa  event*  war*  conducted  to  alaulata  tha  movement*  of  th*  anaay. 

Th*  appearance  of  each  targat  array  was  accomplished  in  a  two  phaaa  operation. 

Tha  us*  of  this  phasing  allowed  th*  targat  arrays  to  b*  exposed  initially  to 
test  th*  first's  ability  to  datect  targat*  at  varying  ranges.  Pirars  ware 
Instructed  to  fir*  at  targets  once  detected.  If  tha  targets  were  not  detected 
during  th*  initial  phasa,  a  period  of  "forced  detection"  was  included  to  enable 
the  evaluators  to  determine  tha  fir*  units'  ability  to  placa  and  adjust  fire 
on  a  detected  targat.  Th*  phasing  of  tha  targets  was  as  follows i 

(1)  Phase  I.  This  phase  was  a  silent  period  in  which  targets  appeared 
at  approximately  thraa  second  intervals  until  they  ware  all  exposed .  It  was 
during  this  phasa  that  target  detection  capability  was  determined. 

i  (2)  Phasa  II.  Thie  phaaa  was  initiated  by  a  forced  detection  sequence 

I  in  which  selected  targets  were  programmed  to  fir*  their  simulator.  This 

|  insured  that  the  targets  had  been  detected  so  that  ability  to  place  and  adjust 

*  fir*  could  be  measured.  Tha  phase  consisted  of  both  target  exposure  and 

f  simulator  fire.  Targets  during  this  phaaa  war*  prograasaed  to  nova  up  and 

down  at  different  Intervals  while  displaying  varying  degrees  of  simulator 
%  fir*.  The  amount  of  targat  exposure  and  simulator  firs  ware  based  on  the  dia- 

5  tanca  the  target*  were  located  from  the  friendly  element  in  an  attempt  to 

f  appro* -h  combat  realism  within  the  constraints  of  th*  experimental  design. 

t 

(3)  During  Phase  I,  if  a  flrar  was  able  to  detect  a  target,  to  fir*,  and 
to  obtain  a  hit  or  near  miss,  tha  Phasa  waa  terminated  and  Phaaa  II  was  auto¬ 
matically  initiated.  If  a  hit  or  near  miss  did  not  occur,  Phaaa  II  was 
■  initiated  automatically  upon  completion  of  tha  time  programmed  for  Phase  I. 

c.  Design  Considerations.  In  planning  statistical  aspects  of  tha  experi¬ 
mental  design^  factors  and  constraint*  considered  vara  tha  following! 

(1)  Basle  of  Issue  (BOI)  Tha  main  object  of  the  experimental  design 
mas  to  allow  comparisons  of  selected  STANO  device  sixes  in  a  rifle  squad,  rein¬ 
forced  with  a  3  man  machlnagun  section,  in  terms  of  thalr  firs  effectiveness. 

Th*  different  mixes  of  STAUO  devices,  BOI 'a,  v« r*  determined  on  th*  basis  of 
military  rationale  so  that  reasonable  differences  In  the  BOI 'a  could  be 
detected . 
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(2)  Light  Levels .  Light  intensity  wu  of  primary  concern  la  the  SOX 
comparisons,  lather  than  carry  light  as  a  continuous  coverlets,  it  was 
consider ad  aaalar  analytically  and  operationally  to  categorise  It  into  three 
levele  as  follows i 

High  Light!  Ambient  light  >_  2.0  x  10-3  footcaodlas  , 

Hid  Light!  2,5  x  10”^  footcaodlas  <  Ambient  light  «  5.0  x  10~J 

Low  Light!  Ambient  light  <  2,5  x  lC"*  footcandlas. 

In  the  time  frame  set  for  the  experiment  the  first  aid  light  occurred  on  the 
evening  of  10  July  and  ended  29  July,  high  light  began  15  July  and  ended 
2A  July,  and  consistent  low  light  first  occurred  26  July  and  continued  Into 
the  first  part  of  August.  It  was  avldant  from  this  that  simple  randomisa¬ 
tion  of  light  levels  over  trials  could  not  have  bean  dona. 

(5)  Tima.  Since  light  laval  occurrence  was  dictated  as  noted  above, 
the  completion  of  the  experiment  was  restricted.  If  the  record  trials  ware 
not  completed  in  the  time  allotted  the  experiment  would  have  to  have  been 
extended  until  the  needed  light  level  reoccurrsd.  Results  were  also  needed 
as  soon  as  possible  to  assist  in  a  decision  relating  to  the  production  of 
the  selected  STAND  devices  under  test. 

(4)  Squads.  Drawing  upon  available  personnel, it  was  decided  that  12 
squads  of  13  man  would  ba  used  in  the  experiment.  An  attempt  was  made  to 
balance  the  squads  with  respect  to  military  background  (p.g.  combat  versus 

|  non  combat)  and  human  factor  considerations  (e.g.  playerp  wearing  glasses 

versus  not  wearing  of  glasses).  Although  the  attempt  was  made  to  balance 
the  squads,  it  wee  realised  from  previous  experimentation  that  large  varia¬ 
tion  in  the  performance  of  the  squads  could  occur. 

(5)  Sample  Slsa,  Considering  the  above  factors  and  constraints,  it 
was  decided  that  four  BOZ’s  would  be  used  in  the  experiment,  appesrlng  with 
each  of  the  three  light  levels.  The  bsslc  experimental  matrix  than  constated 
of  12  cells.  The  need  for  12  cells  in  the  experimental  matrix,  ea  well  as 
results  from  previously  conducted  night  live  fire  experiments,  dictated  s 
sample  else  of  12  observations  per  cell  of  the  design  matrix  in  ordar  to 
obtain  raaeonabla  probabilities  of  Type  I  end  Type  II  errors.  The  number  of 
light  levels  end  BOi's  and  the  sample  sice  determined  the  experimental 
matrix  balowi 


Light  Level 

BOI  1 

BOI  2 

BOI  3 

BOI  4 

Low 

12 

12 

12 

12 

Mid 

12 

12 

12 

12 

High 

12 

12 

12 

12 

m 


(6)  Learning /Motivation.  Because  of  the  abov«  time,  player,  and  sample 
alaa  consideration* ,  It  vaa  decided  that  the  main  experiment  would  begin  on 
10  July  end  trials  would  be  conducted  every  night  until  completion  on  30 
July,  The  number  of  players  and  the  tins  available  to  complete  the  experi¬ 
ment  dictated  that  the  playere  would  be  uied  repeatedly  for  a  period  of  time 
which  Included  weekenda.  Twelve  trlala  were  scheduled  for  each  squad.  It 
was  thought  that,  with  the  players  continually  repeating  the  trlala,  they 
would  learn  when  and  whare  to  fire  to  obtain  a  target  hit.  It  was  elec 
anticipated  that  the  players'  motivation  to  parform  well  would  decline 
as  the  number  of  repeated  trials  Increased.  These  carry-over  effects 
(learning  and  motivation),  which  would  have  been  counfounded  with  light  levels 
and/or  BOls  If  a  simple  randomisation  of  BOIa  over  or  within  light  levels 
had  bean  uaad,  were  a  aaior  consideration  In  the  aelection  of  the  statistical 
design. 

2.  DESIGN  STRUCTURE  -  HOD EL. 

a.  Design  Layout.  The  statistical  design  selected  for  this  experiment 
was  basically  a  split-plot  design  where  light  levels  corresponded  to  whole 
plots.  Twelvo  squads  were  formed  and  randomly  divided  into  three  groups  of 
four  equads  each.  One  group  fired  only  during  low  light,  another  during  mid 
light  only,  and  the  third  group  only  at  high  light.  Within  a  light  level, 
three  factors  were  examined.  These  were  B0I/AMM0  Mix  combinations,  Squeda, 
and  Blocks.  For  this  experiment,  e  block  was  defined  as  a  group  of  four 
trials  in  which  each  squad  fired  once  and  each  B0I/AMM0  Mix  combination  was 
uaed.  The  three  factors  were  run  In  three  Latin  Square  arrangements  where 
each  squad  flrad  each  BOI/AMMO  Mix  combination  three  times  for  a  total  of 
twelve  trials  per  squad.  The  first  Latin  Square  under  mid  light  had  squads 
1.  2,  3,  and  4  as  rows,  blocks  1,  2,  3,  end  4  as  columns,  and  BOIa  1,  2,  3, 

and  4  as  the  randomly  assigned  treatments  within  the  square.  The  second 

Latin  Square  under  the  same  light  level  had  the  same  squads  aa  rows,  blocks 

S,  6,  7,  and  B  as  columns  and  the  same  BOIs  again  randomly  placed  in  the  rows 

and  columns.  The  third  Latin  Square  under  the  aid  light  level  continued 
in  the  same  manner.  This  method  of  randomization  was  used  for  each  light 
level,  squads  5,  6,  7  and  8,  being  used  for  high  light  and  squads  9,  10,  11 
and  12  being  used  for  Inw  light.  It  should  be  noted  chat  since  the  block 
number  (1  through  12)  corresponded  to  the  number  of  times  each  squad  fired 
in  a  given  light  level,  thie  block  effect  was  closely  related  to  both  moti¬ 
vation  and  learning.  The  experiment  was  expressly  designed  to  balance  block 
and  squad  differences  in  order  to  minimize  their  counfounding  effect  on  the 
BO I  and  light  level  comparisons.  Additionally,  the  design  enabled  Che  com¬ 
pilation  of  useful  information  as  to  the  nature  and  extent  of  eho  block  nnd 
squad  dlfferencaa. 


A 
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b.  Model.  The  eteclecicel  model  for  the  beelc  deelgn  wee 

’nil  -  “  ♦  Li  *  si(l)  «“■>*  ♦  +  ‘lju 


where 


end 


'ijkl 


u 

L. 


J(l) 

Bk 

BA, 


-  observation  corresponding  to  the  1th  light  level,  jth 
squad  In  the  ith  light  level,  kth  block,  and  1th  basis* 
of- issue/ ammo  mix  combination 

“  true  overall  mean 

■  1th  light  level  effect,  1  ■  1,  2,  3 

“  Jth  squad  effect  in  1th  light  level,  J  -  1,  2,  3,  4 

•  kth  block  effect,  k  -  1,  2,  ...  ,  12 

*  1th  baals-of-issue/ammo  mix  combination  effect, 

1  -  1,  2,  3,  4 


(BxL'>lk  -  block  by  light  level  interaction 

(BAxL)  -  baals-of-issue/ammo  mix  combination  by  light  level 
11  Interaction 


Lljkl 


•  random  error  corresponding  to  the  y^^  observation. 


L1  “  °*  SJ(i>‘ 


12 


N<°,  \)\  l  Bk  -  0 


k-1 


l  BA  -  0;  l  (ExL)  -  l  (BxL)  .  -  0; 
1-1  i  k  1K 


I  (BAxL)  -  y  (BAxL)  ..  -  0; 

1  1  x 


euki  * N(0*  V*  e(sj(d)  "  s^ijki) 


0. 


It  was  felt  that  this  model  would  fit  the  observed  duta  reasonably  well  and 
that  the  underlying  assumptions  of  the  model  were  justifiable. 


3.  ANALYSIS. 

a.  Residual  Analysis.  Vhe  model  was  fit  to  the  observed  data  for  each 

of  the  MOE’a. (1  -  arcsin  / x  was  used  for  THSAE),  Analysis  of  the  realduals 
showed  the  luodel  and  corresponding  ussuaptlons  to  be  reasonable, 

b.  Analysis  of  Variance.  The  anelysis  of  variance  table  used  in  the 
analysis  A  the  observed  date  corresponairg  tc  the  model  described  above  was: 


f  7 
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The  mui  and  virltnen  of  respective  coll*  and  call  differences  vara  aa 
follows i 

(1)  Moan  of  observations  at  1th  light  laval,  1th  BOl/Aaao  six 
»l..l  '  "  +  li  +  *<!)  *  **1  *  <*“>U  * 

v“  •  h !»«+ 

(2)  Difference  in  two  call  Mans  (different  BOI/Asno  mix)  at  same 

light  laval  (1  1') 

*i..l  ~  h..l'  "  (BA1  *  «!•>  +  "  <BA*L>il’J 


+  **l..l  "  *l..l'^ 

2o2 

v*r  (?1..1  ■  ^..l0  ““lT 

(3)  Difference  In  two  cell  means  (different  BOI/A tme  mix)  at  different 
light  levels  (1  ft  l' ,  1  ft  1* ) 

*i..i "  yi*,.i’  “  ai  ■  li*)  +(BAi  ‘  “r5  + 1 (baxll) tl 

-  <BAxLL)1,1,]  ♦  (S(1)  -  S(if))+  (o1>#x  - 

Var  (*i..l  “  V..lf)  "  ?  <0e  *  3°s) 

(4)  Mean  of  BOI/Asmo  mix  averaged  over  light  levels 

v“  3°i> 

(3)  Mean  difference  of  same  BO I /Ammo  mix  at  different  light  levels 
(1  ft  1’) 

*l..l  "  V..1  “  (L1  "  h’*  +  <5(1)  “  S(i’))+  IBAj,Lt)il 

-  (BAxLL)l(l)  +  (e4  a  -  e^^) 

Var(5i..i"?i \.i)mr  <°l  +  3a s2) 
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4.  RESULTING  ADVANTAGES  OF  TBS  DESIGN  USED.  The  ability  of  CDs  design 
to  make  use  of  the  Latin  Square  arrangement,  where  squads  and  blocks 
(measure  of  learning/motivation)  fora  ad  tha  rowa  and  columns  of  Cha  square, 
waa  an  important  advantage  of  Its  selection.  Extending  blocks  over  the  three 
squares  in  a  given  light  level  gave  the  advantage  of  yielding  information  on 
learning/nocivation  over  a  greater  period  of  tine.  The  design  then,  not 
only  reduced  cha  error  naan  square  by  removing  squads  within  light  levols 
end  blocks  ae  sources  of  error  but  also  gave  insight  into  the  variability 
anong  squads  and,  aost  important,  it  gave  information  aa  to  the  effect  of 
laarn’ng/aotlvatlon  on  players  who  were  used  over  extended  periods  of  tine. 
Another  advantage  of  this  design  was  that  it  lent  itself  to  continuous 
analysis  of  the  data.  Upon  the  completion  of  a  square,  analysis  could  be 
done  and  in  this  way  the  data  could  be  aonitored  and  analysed  in  auccesalve 
steps. 
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Suamary 

When  estlaatlng  the  parameters  of  a  tine  series,  one  ordinarily  has 
available  an  entire  set  ot  time  series,  or  a  multiple  series.  Some 
obvious  examples  are  economic  and  serologic  series.  This  paper  treats 
the  situation  In  which  the  time  series  of  interest  Is  one  of  a  set  of 
observed  series  whose  parameters  are  "generated"  Independently  by  a 
random  mechanism,  i.e.,  Independent  realizations  of  a  random  variable  & 
with  unknown  (prior)  distribution.  This  suggests  the  Empirical  Eayea 
approach  to  the  estimation  problem.  The  specific  models  studied  are 
the  first  order  auto-regressive  process  with  aero  mean,'  and  the  time 
aeries  regression  model  with  auto-correlated  error  term.  Although  exact 
Empirical  Bayes  estimators  do  not  exist  for  the  various  situations, 
approximate  Empirical  Bayes  estimators  ate  developed  and  used  to  obtain 
smaller  mean  squared  error  than  with  the  usual  estimators. 


1.  Decision  Theoryi  Bayes  and  Empirical  Bayes 

Let  un  suppose  that  we  are  Interested  in  estimating  a  parameter  e, 
end  that  this  estimation  situation  occurs  repeatedly  for  different  para¬ 
meter  vslues.  Further,  suppose  that  Che  specific  (but  unknown)  values 
of  3  are  Independent  realizations  of  a  random  variable  6,  with  distri¬ 
bution  C ) .  The  observations  x^,  x^,.,.,  which  are  used  to  estimate 

0  follow  ths  conditioned  distribution  f(x|e).  The  decision  (lp  this 
case  our  estimate,  0)  Is  characterized  by  a  lose^function  1(9,3).  This 
function  tepresents  our  "lose"  when  we  sey  that  6  Is  the  parameter  value, 
when  Its  value  Is  6,  In  this  situation,  we  are  naturally  Interested  In 


finding  an  estimation  procedure  which  minimizes  our  lost.  Wa  thus  saak 
that  procadura  a  which  will  Minimize  our  overall  expected  loss,  or  regret. 

£g($)  ■  /  £  l(e,a)  h  (x.a)  dedx 

•  /  /  l(e.$)  f  (x|e)  g  (a)  dedx, 

*  0 

where  gfe  )  ■  G'(e )  and  h(x,«)  Is  the  bivariate  density  of  x  and  a.  If 
8  represents  the  "minimizing  procedure,"  l.e., 

Es(e)  ■  m^n  E6(^),  then  5* Is  called  the  Bayes  estimate  of  • 
and  Eg(a)  Is  the  Bayes  regret. 

The  most  widely  used  loss  function  In  estimation  is  the  quadratic  loss, 
1(e,$)  •  tc(e-tl)2.  This  yields  the  very  appealing  Bayes  estimator, 


9  »  E(e|x)  with  regret  E[Var  (a|x)]. 

Let  us  consider  the  special  case,  when  e-Nfw.o2)  and  x  -N(e,o2). 
For  the  Quadratic  loss,  we  obtain, 


E(e|x)  - 


xiiUsai 

7 TVT 


Notice  that  f(x) 

£(•!*) 


I 

i 


f(x|a)g(e)de  ,  and 

ef(x|e)g(a)de 

f(x) 
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It  Is  apparent  that  If  tha  density  function  g(0)  Is  not  known, 
then  It  Is  Impossible  to  obtain  £(e|x).  For  the  special  case, 
*-«(•, a2),  E(«!x)  can  be  rewritten  as: 


E(a|x)  »  x  ♦  o2.  ,  where 


f-(«)  ■  . 


The  marginal  density  f(x)  Is 


not  known  exactly,  but  It  can  be  estimated  with  several  past  obser¬ 
vations  from  several  different  (random)  values  of  a.  In  that  cate  we 


/\  A 

E(e|x)  ■  x  ♦  a2. which  Is 


an  approximation  to  the  Bayes  estimate  called  the  Empirical  Bayes 

estimate.  The  density  ratio  In  the  right  hand  side  of  equation  (1) 

can  be  estimated  well  by  estimating  numerator  and  denominator  separately. 

(See  Parser.)  Under  certain  mild  conditions,  the  risk  Incurred  by  using 

the  Empirical  Bayes  estimator  approaches  that  of  the  true  Bayes  estimator  as 
n,  the  number  of  experiences  Increases. 

2.  The  First  Order  Auto-Regressive  Parameter 
The  simplest  auto-regressive  model  Is: 

x^  ■  «{,  where  the  ct  ore  (2) 

Independent  errors.  If  tha  ara  assumed  normal,  than  tha  M.L.B.  of  •  la: 


1- 


I 

n 


It  has  been  shown  (Mann  &  Wald,  1943;  and  Durbin,  1950)  that  if  |«|<lthao 
«  •  -«)  Is  aspnptotlcally  N(0,  l-«2)  regardless  of  the  dlstrl- 

O 

butlon  of  the  «t.  The  likelihood  function  L"  ,^f(x^|«,o)  can  be 
written 

— {ns2  *  («-«)2  tx 
2o* 


-n/2  “ 

L  ■  (2ioz)  exp 


which  Is  of  the  general  form: 


f(x|-,o2)  «  f{e.s2,u|«,c2), 

where  u  •  lx2  ^  and  x'  ■  (xj  ,x2,.. . ,xn).  It  can  be  shown  that 
Edx2^)  -  E(E-2tcn_t)-^o2/(1-2), 


and  thus 

L 

so  that  «  Is  "asymptotically"  sufficient  for  «.  For  small  samples 
we  have, 

f(|l«»«2)  ■  f(«.s2,u|«,e2).  But 
■  f(x|-.o2)  •  gl-,®2) 


(2»oz) 


[•-)2/(l-2)>  . 
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* 


•  f(«,S2.u  !•»•*).  9(*»«2)- 
f(*}  ■  //  f(«.s2,iiKo2)  .  «(«t«2)  ,  d^a2 


*  o 

f(«,s  ,u).  Therefore 

f(  «to2\) 
f(x) 


f(< 


2  *  p 

/!*,$  ,«),  and  after  Integrating  w.r.t. 


o  ,  w«  obtain. 


f(«|x)  •  f(«|«.s2,u).  Note  that  E(«|x)  ■  E(«|«,s2,u)^E(«|«). 

We  shall  Ignore  the  dependence  of  Var  («)  on  «  and  propose  to  use 
a  “marginal"  empirical  Bayes  estimator  (Cleamr  and  Krutchkoff,  1968) 
for  «. 

Assume  that  «  Is  a  random  variable  with  unknown  density  function 
g(«) ,  and  that  we  have  observed  k>l  time  series  Involving  the  Indepen¬ 
dently  generated  parameters  *i»*2****»*jc»  If  have  computed  the 
M.L.E.'s  «j,  “2 *•  * ’ '"k  th8n  "wnjlrol  empirical  Bayes  estimator 
(Cleaner  and  Krutchkoff,  1968,  and  Rutherford  and  Krutchkoff,  1969) 
which  we  propose  for  Is: 


Ek(‘kl*k)  "  *k  +  {  (1**k)/n} 


W 

fk(%) 


where 


(4) 
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and  vher* 


+  h)/2h 


Ci  ’  (\  *  *1 

Fi  •  c\  -  V/2h  > 

h  -  k'1/5  M*xt(£  -  i)2  >1/2,  {(1-  *«J)/n>1/2J, 

and  a  »  length  of  preaaat  (k(b)  time  aariaa. 

3.  Prediction  of  future  value*  of  Auto-Regreaaiva  Model. 

The  A.R.  aodal  la  uaeful  In  predicting  future  obaervatlon*  if  a  good 
eatlaate  of  «  la  available  alnce, 

Vi  -  "  *t  +  et 

and 


*<Vil*>  "  *  “t  • 

Aaauae  that  ,  la  an  unknown  par an* ter,  9,  and  that  wa  dealre  to 

mlnlnlaa  tha  loat  function  L(e,6)  -  (6  -  8)  .  furthermore,  aaauna  a  multi 
varlata  diatrlbutlon  f(x,<*,6).  Let  f  repreaant  the  denalty  function  for 
the  aaaoclatad  random  variable*.  Formally,  then,  the  Bay**  eetlnate  of 

6  1,1  //9f(*,«,fl)d9d« 

8(6  |x)  “  7rifsr,~<w~ 

•  //0f(e|x,-)  d8d« 

■  I!  9f (8|x,«)g(  «|x)d9d« 

<« 


•  /g<Hx)»*(«|x,«)d" 


•  /|(«|*>  •«  *  d« 
«  * 

•  *(«|«)  •  *t 


Thus,  to  minimise  the  squared  error  of  the  estimate  of  x^j,  It  io 

only  necessary  to  use  tho  Bayes  estimate  of  B(-|x)  in  tho  uouol  estimator. 
Xa  other  words,  to  estimate  tho  future  value  x  ,  well,  just  estimate  the 
parameter  «  well. 

4.  The  Regression  Model. 

The  A.R.  process  may  appear  as  tho  error  term  in  regression  models. 
(Suppose  that  x^,  Xj,  Xj, . . .  are  fixed  and  known  nuabera  (Investment 

expenditures  for  example),  sad  ^  ««  u  ^  ♦  ct  is  an  A.R.  process.  Then 

yc  e  B»t  +  u  1»  called  a  time  aerlaa  regression  model  with  auto-regra* 
alva  error  term.  The  model  can  ha  written  (S) 


yfc  “*  yt-l  "  ®^*t  “*  *t-l^  +  ct 


yielding  theJM.L.B.'e  for  <%B: 

[  (yt  -  Bxt><yt_i  -  fa  ) 

A 

<X  m  •  •  i 

lmi  <yt,x  -  K-i>2 

l  <x  -«  X  ,)(y  -«  y  .) 

„  t-1  c  1  1  _ „ 


n 

I 

t-1 


I  <xt  Vi)2 


(6) 


(7) 


while  the  estimate  of  o  let 


Dm  M.L.B. '•  fox  «  and  0  (6),  (7)  are  non-linear  and  require  an 
iterative  scheme  for  ooluclon  (Cochran  ft  Or cut t,  1949)*  It  nay  ba  shown 
that* 

*<o2|  «*e)  •  o2, 

B(S|«*9)  ■  9*  *»d  using  tho  raauit  of  Mann  ft  Wald* 

B(*| «,B)  »  “  asymptotically. 

Furthermore,  if  ct  -  MID  (0,o2)  than  6  *  M{B,o2/E(*t  -  e*^)2}. 
n  , 

hot  q(«,b)  -  l  tyt  —  yt-1  -  B(xt  -  «  *t_L>]  • 

^*c  ■  £<<* t  Vi +  ct*tI- 

Mow  if  B(u  )  •  B(e  )  ■  0*  wo  nota  that 
o  o’ 

n  .  , 

un  •  l  ^  «1,  ao  that  BO  <J^  «3  B)  •  0. 

1*0 

Thus,  we  shall  assuaa  that  «  and  8  ora  independent. 

Note  tha*t  8  is  sufficient  for  6  whan  «  is  known  but  that  «  is  not 
eufflclent  for  «  when  fi  is  known.  When  «  and  6  are  both  unknown,  then  the 
likelihood  function  Involves  ssMoral  statistics.  Thus  if  y  •  (ylty,,...,y  ), 
the  likelihood  function  will  taka  the  fora*  n 

f  (*»B»u|  **B,o2). 

Mow  suppose  that  «  and  B  are  randoa  variables  with  joint  density 
g(**8),  and  that  the  estimation  problea  has  occurred  k-1  tlaes  previously. 

It  nay  ba  shown  that  the  posterior  densities  for  «  and  6  nay  ba  writ tent 

f(«|y)  •  f  («|  «,S,a2,u) 

end  f(l|y)  •  f (B| «,B  ,s2,u)  . 

We  therefore  propose  to  uaa  "marginal"  estimators  for  «  and  6. 
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i 

t 


Than, 


V\l  V  "  ;+(<l  *  “2>/n}#{  ^ 


where 


sar  •  J,  (  * 


W  *  m 

*;<v-  tfk(%^)-fk(ik>>/h. 

h  -  k"1/S  Mu  {{£  -  »)2)1/2,  {(1  -«  2>/n)l/2], 

•ad  n  •  length  of  kth  tinea  aeries,  sod 

.  £k  <v 


VBki0V  -  ^  +  —  — 


?  /  -  fv<8.  ) 

I  <xt  ■  Vt-1}  k  k 

t-1  z  1 

h  -  k“1/5  Mu{[i  [  (Bt  -  S)2]1/2,  .)  . 


5.  Prediction  with  the  Regression  Model, 

As  In  the  csss  with  the  A.R,  nodal,  v«  shell  assume  that  ths  future 

value,  yt+1  la  the  paranater  9,  and  that  there  exists  a  distribution 

*<y.0, •,*»).  **»»•*•  Jt  -  bxf  +ut, 

u ^  -  e«t-1  +  £t,  and  are  Independent  errors.  Then, 

yt+l  "  ayt  +  b(*t+l  ‘  “t>  +  et  and 

E(yt+il*'b*yt)  “  **t  +  b(xt+l  "  **•)•  ThBn» 

E(e|yt)  -  /6f(e |y)d9 

•  /0f(e,y)dO//f(6,y)de 
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///Of (e,y,a,b)dadbd8 
"  ///f(8,y,c,l)dadbd8 

•  /// 0  f (8|y,a,b)g(y,a,b)dedadb/g(y) 

-  //B(6|yt,t.b>^^  dadb 

•  //E(6  j  yt , a,b)g (a ,b | y )dadb 

-  //*U,b|y)t*yt  +  b(xf+1  -  Mt))d»db 

-  *u!yc)yt  ♦  B(bjyt)ixt+1  “  E<*|yt>  •  *t) 

•  **t  ♦  £(*t+l  -  •v- 

The  lut  two  lines  follow  If  a  and  b  ara  lndependa.  t, 

6.  Oanaral  Simulation  Procedural  and  Rasulta. 

the  Monte-Carlo  simulations  in  chin  rasaarch  wara  conducted  on  tha 
IhM  300  computer  ae  Virginia  Polytechnic  Znatltuta  and  State  Unlvaralty, 
Blacksburg,  Virginia.  For  each  of  the  estimation  situations,  SO  values 
of  tha  parameter  6  wara  ganaratad  fron  either  a  uniform  distribution  or 
one  of  tha  following  Pearson  distributional  Bell- shaped ,  U-shaped,  J-left 
shaped,  or  J-rlght  shaped.  These  50  values  of  6  wara  uaed  to  simulate  up 
to  50  Independent ,  consecutive  experiences,  Tha  classical  estimate  for 
each  value  of  6  was  obtained,  and  tha  I.B.  estimate  was  computed  for  tha 
2nd,  4th,  6th,  10th,  12th,  25th,  and  30th  experiences,  Tha  entire  pro¬ 
cedure  was  repeated  500  times,  with  50  dlffararct  values  of  6  on  each 
repetition.  The  M.S.B.'s  ware  estimated  for  each  of  tha  past  experience 
numbers  listed  above,  as  follows: 

.  500  „ 

®*S(j)  *  5C5  l  (9i  ~  V* 

5Wi-l  1  1 


BXS  (8) 


.  500 


<*1  -  V 
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For  economy  of  computer  time,  the  errors  were  almost  exclusively  uniform. 
Most  of  the  estimates  Involved  were  "asymptotically"  Independent  of  error 
distribution.  A  number  of  simulation  runs  were  duplicated  with  both  nor¬ 
mal  and  uniform  errors,  but  almost  no  difference  was  noted  In  the  quality 
of  the  empirical  Bayes  relative  to  the  classical  estimator. 

(See  Table  II). 

Some  typical  examples  of  the  relative  Improvement  of  empirical  Bayes 
procedures  over  classical  are  given  In  the  accompanlng  Tables.  As  is  usual 
in  empirical  Bayes,  the  Improvement  is  given  in  terms  of 

-  EMS(8)/EMS(6),  for  k  experiences  and  for  several  different  s 
vlaues  with 

Z  ■  E  Var(6|d)/Var(6)  and  for  each  type  of  prior  distribution.  It 
has  been  found  in  previous  research  that  for  a  fixed  number  of  past  experi¬ 
ences,  R,  invariably  decreases  with  2.  All  of  the  simulations  were  con¬ 
ducted  using  equi-length  series.  The  empirical  Bayes  estimator  does  not 
require  this,  but  the  computer  time  and  storage  were  reduced  considerably  In 
this  way.  Incidentally,  It  was  noted  in  the  simulations  that  the  R  values 
were  not  notlcably  different  for  series  of  length  12  and  36  observations. 

In  the  regression  model,  a  change  in  the  Z  value  of  one  of  the  para¬ 
meters  did  not  change  the  R  value  of  the  estimate  of  the  other  parameter. 
This  is  attributable  to  the  Independence  of  the  M.L.E. 's.  Furthermore, 
the  Improvement  in  R  of  the  e.B.  estimate  of  b  with  respect  to  experiences, 
is  far  greater  than  that  for  a  up  to  about  5  or  6  experiences  and  after 
that,  the  situation  is  reversed.  This  produces  a  "crossover"  effect  in 
the  K-values  of  a  and  b. 

In  all  of  the  situations,  for  Z^.?5,  a  tremendous  Improvement  Is  noted 
lh  R  values  up  to  about  6  experiences'?  Thereafter  there  la  a  "tapering  off" 
in  efficiency  noted.  The  optimum  valne  in  most  situations  seems  tc  be  about 
6  experiences. 
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TABLE  I 


VALUES  OF  E.  B.  ESTIMATES  OF  A.  R.  PARAMETER 
EXPERIENCES _ 2 _ 5 _ TO _ 25 _ 50 


UNIFORM  PRIOR 

Z-  .36 

m 

m 

.96 

.83 

.79 

.73 

.96 

.86 

.81 

.73 

.69 

1.1 

.87 

.74 

.69 

.63 

.58 

5.2 

.8 

.63 

.57 

.49 

.43 

BELL  SHAPE  PRIOR 

Z-  .43 

.97 

.9 

.86 

.8 

.77 

.77 

.93 

.81 

.78 

.73 

.68 

1.6 

.86 

.72 

.68 

.61 

.55 

4.6 

.83 

.65 

.59 

.49 

.41 

U-SHAPE  PRIOR 

Z-  .23 

m 

m 

.94 

.89 

.86 

.42 

1.0 

.93 

.87 

.83 

.8 

.9 

.93 

.81 

.76 

.74 

.7 

2.6 

.78 

.67 

.6 

.54 

.5 

J-SKEWED  LEFT  PRIOR 

Z*  .36 

. 

m 

.94 

.84 

.78 

.73 

.97 

.88 

.81 

.73 

.7 

1.65 

.89 

.73 

.67 

.61 

.55 

4.9 

.83 

.61 

.55 

.46 

.4 
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TABLE  II 


R  VALUES  OF  REGRESSION  PARAMETERS  FOR  M»12 


Experiences 


Uniform  Errors  Nomil  Errors 


9 

.957 

.890 

C 

4 

.m 

.760 

(L 

.811 

.710 

© 

10 

.790 

.653 

1  w 

2* 

.714 

.618 

bW 

SO 

.644 

.563 

z 

2.9 

3.8 

a 

b 

.954 

.895 

.873 

.770 

.812 

.711 

.786 

.665 

.715 

.622 

.646 

.592 

2.9 

3.8 
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TABLE  III 

R-VALUES  FOR  THE  REGRESSION  MODEL  PARAMETER  ESTIMATES 


PARAKTER  *  -  UNIFORM  PRIOR 


EXPERIENCES 

2 

5  10 

25 

50 

l.  » 

.48 

.94  .92 

.85 

.8 

ft 

.91 

.95 

.87  .82 

.76 

.72 

3.74 

.85 

.73  .65 

.57 

.5 

PARAMETER  6 

-  UNIFORM  PRIOR 

.47 

.90  .88 

.85 

.81 

V 

1.05 

• 

.84  .78 

.73 

.68 

1.67 

- 

.82  .71 

.62 

.57 

5.8 

1.0 

.67  .56 

.51 

.45 

PARMCTER  a 

-  BELL  SHAPE  PRIOR 

za  • 

1.28 

.88 

.78  .73 

.69 

.65 

• 

5.25 

.84 

.71  .63 

.53 

.47 

PARAMETER  b 

-  BELL  SHAPE  PRIOR 

2h  ■ 

2.39 

m 

.74  .65 

.58 

.51 

D 

8. 35 

.88 

.61  .54 

.5 

.46 

PARA1CTER 

a  -  U-SHAPE  PRIOR 

z«- 

.675 

.92 

.86  .82 

.8 

.78 

PARAMETER 

b  -  U-SHAPE  PRIOR 

V 

1.22 

1.0 

.81  .73 

.65 

.6 
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SOME  EFFECTS  OF  AN  IMPROPER  SCREENING  TECHNIQUE 
ON  THE  AOQ  WHEN  USING  CSP-i 


Frmd  L.  Abraham 

U.S.  Army  Amu  alt  ion  Procurement  and  Supply  Agency 
Jo  Hat ,  Illinois 


1.0  INTRODUCTION 

1.1  Continuous  sapling  procedures  art  accaptanca/ractif icatloa  inspsction 
plans  developed  for  usa  where  production  is  continuous  and  where  tha  formulation 
of  inapactlon  lots  for  lot-by-lot  accaptanca  say  bs  impractical  or  undaslrabla 
(such  as  in  ths  inspsction  of  aaunitlon),  or  artifical  as  la  of  tan  the  casa 
with  convsyor  11ns  production.  While  there  exist  many  continuous  sampling  pro¬ 
cedures,  they  all  have  similar  characteristics;  namely,  inspection  is  carried 

out  by  altarnata  sequences  of  consecutive  item  inspection,  called  10QX  inspection 
or  screening  and  seqoencea  of  production  from  which  only  samples  are  taken.  The 
objective  of  continuous  sampling  procedures  is  to  provide  assurance  that  the 
long  run  percentage  of  defective  unlta  in  accepted  product  will  be  held  down  to 
some  prescribed  limiting  value  which  will  not  be  exceeded  no  matter  what  quality 
of  product  is  submitted  to  inspection.  This  limiting  value  is  called  the 
Average  Outgoing  Quality  Limit  of  simply,  the  AOQL. 

1.2  The  simplest  and  perhaps  the  most  widely  used  continuous  sampling 
procedure  is  Dodge’s  CSP-1.  It  la  applicable  only  to  quality  characteristics 
subject  to  non-destructive  inspection  and  on  a  go-no-go  basis  -  that  la,  classifi¬ 
cation  of  units  is  either  good  or  bad.  Ths  procadurs  is  defined  by  two  parameters, 
f,  the  sampling  frequency,  and  i,  tha  clearance  number  which  is  the  numbar  of 
consecutive  good  units  that  must  be  found  defect  free  before  sampling  is  to  be 
resumed.  Figure  1  gives  the  procedural  flow. 

1.3  For  given  values  of  f,  1  and  soma  fraction  defective  of  incoming  pro¬ 
duct,  there  will  result  for  product  of  a  statistically  controlled  process  a 
definite  average  outgoing  fraction  defective,  called  the  AOQ.  A  statistically 
controlled  process  is  defined  here  as  Bernoulllan,  that  is,  each  unit  has  an  equal 
probability  of  being  defective.  For  these  values  of  f  and  1,  the  AOQ  will  have 

a  maximum,  or  limit,  at  soma  particular  fraction  defective  of  incoming  product. 

As  noted,  earlier,  this  maximum  is  tha  AOQL. 

1.4  There  ere  3  other  properties  besides  the  AOQ  and  AOQL  which  car  bs 
useful  in  describing  a  particular  sampling  plan.  Those  are: 

u,  tha  expected  number  of  units  Inspected  on  screening 
following  the  finding  of  a  defect. 

v,  the  expected  nimber  of  unlta  passed  on  sampling 
before  a  defe'*  i«  found,  and 


This  paper  was  reproduced  photographically  from  the  a>  •*.  lanuscrlpt. 

Preceding  page  blank 


87 


F,  the  ivirt|«  fraction  of  total  product  lnapactad. 

Appendix  A  gives  tha  mathematical  fonnulaa  for  CSP-1. 

1.5  loapactlon  under  CSP-1  can  ba  performed  either  by  judging  e  cleee 
of  defect*  or  by  Judging  individual  defects  independently.  A  cleee  of  defect* 
is  treated  in  the  inspection  operation  as  if  the  claaa  of  defects  were  a 
single  entity  -  that  is,  the  finding  of  any  one  of  the  listed  defects  of  the 
class  causes  screening  to  be  initiated  for  all  listed  defects  of  the  class. 

When  I  refer  to  a  class  of  defects,  I  am  not  making  specific  reference  to 

the  classification  of  dafacts  according  to  there  seriousness  such  as  critical, 
major,  or  minor  classes  as  found  in  Military  Standards  and  Handbooks  on  inspec¬ 
tion.  An  example  of  inspection  by  a  class  of  defects  would  be  the  inspection 
of  a  cylinder  for  three  quality  characteristics  -  inside  diameter  -  outside 
diameter  -  and  length  of  cylinder.  Screening  would  be  initiated  for  all  3 
characteristics  if  any  one  of  the  three  were  found  to  be  nonconforming  or  defect¬ 
ive.  As  might  be  expected,  when  inspection  is  by  a  class  of  defects,  inspection 
during  the  screening  operation  is  sometimes  Incorrectly  limited  to  those  character¬ 
istics  triggering  the  screening  requirement. 

1.6  Since  the  choice  of  f,  i  end  AOQL  to  be  used  to  Inspect  product  Is 
often  based  on  whether  Inspection  is  to  be  by  e  class  of  defects  or  on  an  indi¬ 
vidual  baala,  ve  are  concerned  with  the  effects  of  this  improper  screening  techni¬ 
que  on  the  everage  outgoing  quality.  The  purpose  of  this  paper  Is  to  demonstrete 
the  effects  on  the  AOQ,  end  therefore  the  AOQL,  when  the  Improper  technique  is 
used, 

2.0  SUMMARY 

2.1  The  effects  of  the  improper  screening  technique  on  the  AOQ  will  be 

demonstrated  by  using  the  following  CSP-l  plans  from  MIL-STD-1235: 


(*) 

Code  J 

i  -  200 

f  -  1/100 

AQL  -  1.0X 

AOQL  -  1.35X 

(b) 

Code  K 

i  -  65 

f  -  1/200 

AQL  -  4.  OX 

AOQL  -  4.96X 

(O 

Code  H 

1-15 

i  -  1/25 

AQL  -  10Z 

AOQL  -  10.70X 

Using  these  plans,  thre^  sets  of  conditions  are  considered; 

Case  I  All  defects  in  the  class  are  considered  to 

have  equal  probability  of  occurrence. 

Case  II  Ona  apaciflc  defect  has  twice  the  probability 

of  occurrence  of  any  of  the  othera. 
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44. 


Case  III  On*  specific  defect  tau  can  tin*  the  probability 

of  occurrence  of  any  of  tha  others. 

For  damonatratlon  purposes  it  waa  arbitrarily  dacldad  to  consider  having  flv* 
dafaeca  in  tha  olaaa. 

2.2  AOQ  curvaa  for  tha  thra*  plana  with  tha  thraa  conditions  Impost* 

vara  conserve tad  to  danonstrata  visually  tha  diffarancaa  In  AOQ  whan  using  tha 
correct  scraanlng  proeadura  aa  opposad  to  ualng  tha  lapropar  tachnlquaa. 

Thaaa  curvaa  ara  exhibited  aa  Figures  2,  3,  and  A. 

3.0  DISCUSSION 

3.1  For  ralatlvaly  snail  valuaa  of  tha  Incasing  fraction  daf active  of 
aubnlttad  product,  p,  within  a  particular  inspection  plan,  thara  la  little 
difference  in  affect  upon  the  AOQ  between  correct  and  improper  techniques. 

Aa  p  increase* ,  tha  difference  in  effect  upon  the  AOQ  between  the  correct 
and  Improper  technique  becomes  quite  significant.  It  is  at  these  larger 
values  of  p  where  the  most  concern  is  focused. 

3.2  Visual  lnapactlon  of  Figure  2,  3,  and  A  shows  that  the  improper 
screening  technique  allows  a  much  larger  percentage  of  defective  material  to 
be  accepted  than  doae  the  proper  technique.  Consider  plan,  (a).  Code  J,  AOQL  - 
1.35X  (See  Figure  2).  If  the  submitted  product  ware  AX  defective,  the  Average 
Outgoing  Quality  would  only  be  .2X  defective  when  the  correct  screening  technique 
is  used.  For  the  same  percent  defective,  improper  scraanlng  technique,  Cass  I, 
would  allow  tha  outgoing  product  to  contain  3.2X  defective  or  16  times  more  than 
would  tha  correct  technique;  Casa  11,  15. 5  times  more;  and  Case  III,  3.7  times 
more  than  tha  correct  technique.  The  same  analysis  will  ylald  similar  results 
from  the  two  other  plans  considered. 

3.3  For  the  three  plans  considered  in  thia  paper,  Teble  I  gives  the 
values  of  the  AOQL,  the  percent  defective  of  the  submitted  product  where  the 
AOQL  occurs,  and  compares  these  to  the  valuea  of  tha  correct  method.  It 
clearly  can  be  seen  that  each  case  of  Improper  screening  has  a  much  larger 
AOQL  velu*  than  the  correct  technique  and  that  thaaa  AOQL  values  occur  at  largsr 
valuea  of  p.  The  effect,  then,  of  the  improper  screening  technique  la  the 
changing  of  the  specified  plan  to  one  with  a  higher  AOQL  which  allows  more 
defective  product  to  pees  inspection. 

3. A  Appendices  A,  F,  C  and  D  contain  the  mathematical  derivation  of 

formulaa  for  each  of  the  conditions  considered.  Definitions  of  symbols  and 
terms,  unless  otharwlsa  atated,  ara  those  given  in  NIL-STD-1235. 
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TABLE  I 


C8P-1  PLANS 


HUB 

■81 

mm 

■HI 

f-  1/15 

AOQL 

p 

AOQL 

D 

AOQL 

D 

Cam  1 

3.71 

5.45 

12,75 

17.06 

24.95 

38.0 

Case  n 

3.25 

4.95 

11. 33 

16.75 

23. 68 

36.5 

Gab*  m 

1.79 

2.25 

6. 30 

8.60 

14.22 

21.9 

Correct  Method 

1.35 

1.75 

4.96 

6.25 

10.70 

16.5 
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APPENDIX  A 


DERIVATION  OP  FORMULAE  POR  THE  AOQ 


A.l  For  given  values  of  i  and  p,  there  will  be  an  expected  average  number 
of  units,  u,  Inspected  following  the  finding  of  a  defect.  For  given  values  of 
f  and  p,  chare  will  be  an  expected  average  number  of  unite,  v,  that  will  be 
passed  under  the  sampling  procedures  before  a  defect  is  found. 

A. 2  The  average  fraction  of  total  product  units  inspected  in  the  long  run 

la  defined  by  Dodge  as  being 

(A  I)  F  -  u  »-f? 

u  +  v 


1  —  n  1 

where:  u  -  ± — -S— 

p  q1 


v  - 


£p 


A. 3  It  is  further  assumed  that 

(a)  the  inspection  operation  Itself  never  overlooks  a  defect 

(b)  each  defective  unit  found  is  removed  from  the  operation  and 
is  not  replaced  by  a  good  unit  (a  departure  from  DoJge) . 

The  average  outgoing  quality  is  related  as  follows  to  the  incoming  quality 
P  = 

(A  II)  AOQ  •  P(i^) 

A. 4  Now  consider  u,  the  average  number  of  units  screened  following  the  find¬ 

ing  of  a  defect. 

From  Dodge, 

u  ■  it  ■  >  q  ■  1  "  P 

p  q1 
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or 


(A  III) 


u 


-  1  g  -  p)1 

p  (i  -  p)1 


However ,  vhen  th*  Improper  screening  technique  defined  is  used,  Equetion  (A  III) 

becomes 


(A  IV) 


where 


1  ■ u  ■  *■>;  j 

s  ^  (i  -  s  Ji  >l 


See  Appendix  B  for  explanation  end  derivation  of  Aj^  j 

M  »  * 

VJl  J2 


end  Appendix  C  for 


A.  5  The  expected  averege  number  of  unite,  v,  that  will  be  passed  under  the 

sampling  procedure  before  a  defect  Is  found,  is  not  effected  by  Improper  screening, 
and  remains  as  v  •  1/fp. 

A. 6  Consider  Equation  (A  II), 


AOQ  - 
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Substituting  (A  I)  tor  F  and  simplifying 


(A  V) 


AQQ 


-P  _ 

u  f  p  (1  -  p)  +  1  -  fp 


where  the  valuo  of  u  In  (A  V)  ia  obtalnad  by  using  Equation  (A  IV).  Tha  AOQ 
undar  tha  thraa  conditions  of  improper  scraaning  tachniqua  given  can  now  be 
calculated  using  Equation  (A  V)  after  first  solving  for  u. 

A. 7  Tha  Avaraga  Outgoing  Quality  Limit,  AOQL,  being  tha  maximum  value  of 

the  AOQ  that  will  result  for  any  given  values  of  f  and  i  considering  all 
possible  values  of  p,  can  be  determined  by  differentiating  Equation  (A  V)  with 
respect  to  p,  equating  tha  derivative  to  0,  and  solving  for  p. 

A .7.1  This  was  dona,  but  the  resulting  aquation  in  p  was  found  to  be  too 
complex  for  practical  use;  therefore,  it  is  not  contained  in  this  paper.  Instead, 
Equation  (A  v)  was  used  to  calculate  the  AOQ  for  several  values  of  p,  and  curves 
were  constructed  from  these  points. 
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APPENDIX  S 


DERIVATION  OF  FORMULAE  FOR  Aj'  r'  and  u  ’ 

Ji  *  *Jk  Ji 


I 


B.l  Let  J  -  1,  2, 


J  be  the  defects  listed  In  the  class. 


Po. 


be 


the  probability  that  a  unit  displays  tha  Jth  defect  listed  in  the  class,  and 
&j  the  occurrence  of  a  unit  displaying  the  j1*1  defect,  then  the  probability, 
p,  that  a  unit  displays  at  least  one  defect  in  the  class  is 


(B  I) 


The  relationship  in  (B  I)  can  be  expressed  in  another  Banner  if  we  group  those 
events  with  equal  probabilities  of  occurrences. 


B.2.1  Lat  i  •  1,  2,  .  .  ,  ,  K  be  the  number  of  groups  formed  such  that  there 
are  k  groupings  with  Ji  defects  in  each  group,  and  let  Bj^  denote  the  occurrence 

or  a  unit  displaying  the  Jth  member  of  the  1th  group,  then 


where  +  .  .  .  JR  ■  J 

B.2.2  When  the  improper  screening  method  discussed  in  this  paper  la  used,  it 
becomes  apparent  that  screening  may  be  dona  for  only  one  defect  of  tha  class, 
for  two  defects  of  the  class,  or  for  any  spsclflc  niaahsr  of  defects  listed  in 
the  class.  If  screening  is  done  for  J'  defects  of  the  class,  than  tha  probability 
of  finding  at  least  one  of  the  J*  defects  on  a  particular  unit  is 

(B  III)  p'  -  At’  j'  ,  where 

1  •  •  • 
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B. 3  Case  I 

B.3.1  The  assumption  is  made  chat  each  defect  within  the  claaa  has  an  equal 
probability  of  occurrence,  that  is 

P°j  "  Poj  +  1  *  i  m  **  2 . J_1 

B.3.2  For  this  case  only,  the  general  Equation  (B  II)  becomes 


(B  V) 


■*{$  e4  ■  &  w  \ 


•  1  -  ‘•o.  •  where  *10,  *  1  -  Po 


and  Equation  (B  IV)  becomes 


1  '  U1 
J1  '1 


(BVI)  Aj|  -  Pr 


{&  EJJ 


B. 4  Case  II  and  III 

B.4.1  Case  II  assumes  that  one  specific  defect  has  twice  the  probability 
of  occurrence  of  the  other  four  listed  in  the  class.  This  one  defect  how  has 
a  fraction  defective  2  p0. 
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B.4.2  Casa  III  assumes  that  oaa  apaciflc  dafact  haa  tan  timet  tha  probability 
of  occurranca  of  tha  othar  four  defects  listed  In  the  clast.  The  fraction 
defective  of  this  one  apaciflc  dafact  la  now  10  Pq. 


r 

£ 


B.4.3  Since  tha  specific  defect  in  both  cases  differ  by  the  Constanta.  2  and  10. 
only  one  aat  of  aquations  need  to  ba  derived  from  the  general  aquations.  (B  II) 
and  (B  XV): 


(B  VXI> 


and 


(B  VIII) 


where  p0  •  C  p0.  .  C  •  either  2  or  10  and  the  state  J'  ■  0, 

Jj  1 

J'  ■  0  does  not  exist,  since  screening  would  not  be  reinitiated  unless  at  least 
ofle  defect  were  found  on  the  sampled  unit. 

B.S  For  each  state  Ati  ,  screened  following  the  finding  of  a  defect. 

1  K 


1  -  U  -  Aj’  J')1 

_ 1  •  *  *  K 

A»i  t»  (1  -  A|i 

1  *  *  *  K  J1  * 


or  for  the  cases  presented  in  this  paper 


u 


j: 


l 


j 

i 

i 

3 


■i 

•j 


J 

i 

I 

4 

i 


u 


1 


1 

i 


i 

3 
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APPENDIX  C 


DERIVATION  OF  FORMULAE  FOR  w  ,  . 

J  I  •  •  J 

1  k 


C.  1  It  has  been  ■hows  that  the  improper  sorsenlng  technique  deeoribed 

in  thie  paper  allow#  screening  to  be  dooe  for  one  defect  in  the  olaee,  for  two 
defect#  in  the  cl#*#,  or  for  any  epeolRo  number  of  defeota  Hated.  We  muat  now 
oonatder  the  probability  of  being  on  ecreening  at  each  of  theee  particular  atatea. 

C.  1. 1  Appendix  B  abowed  for  each  of  the  above  atatea.  how  the  expected 
average  number  of  unite,  u  »  ,  ,  ecreened  In  one  aequenoe  waa  found.  Theee 

Jl*  *  ,Jk 

value#  of  u  *  1  must  be  weighted  by  the  probability  of  that  particular  state 

Jl’  *  ,Jk 

occurring. 

C.  1.2  One  would  not  re-initiate  screening  unless  a  defect  in  the  class  were 

found.  Since  a  aampled  unit  may  contain  one,  two  or  all  the  listed  defects.  It  can 
be  aald  that  the  probability  of  going  on  screening  is  the  probability  of  finding  at 
least  one  of  the  listed  defects. 


C.  1. 3  Now  oonsider  the  different  ways  or  states  to  be  on  screening  for  the 

J  defects  listed  in  the  class.  It  follows  that  the  probability  of  being  on  screening 
for  a  particular  state,  denoted  by  w  '  '  ,  given  that  screening  is  in  effect, 

Jr  ’  ,Jk 


(C  I)  w  *  t  .  Probability  of  screening  for  defeota 

Probability  of  screening  for  at  leaBt  one  defect  in  the  class 

where  the  symbols  and  their  definitions  are  the  same  as  those  developed  in  Appendix  B. 


.j&m  ■*W?  ?JW  ~p*| 


f 

r 

r 

s- 

r 

f 


C.  3  lu  Cut  I,  only  where  the  Hated  defeeta  have  equal  probability  of 

occurrence,  the  probability  that  a  a  ample  unit  coo  Ulna  defeeta  In  the  class, 
resulting  In  screening  for  these  j’  defects,  la 


and  the  probability  of  screening  for  at  least  one  defect  in  the  class  is 


Then  (C  I)  becomos 


<CH) 


The  five  weights  for  Case  I  are  then 


W1B 


*Pq  % 


w„ 


10Pq  % 
i-q* 


W3  " 


in  3  2 

1°  p  q« 

o  o 

Tv" 


4 

'’o 


®P«  % 


1  -  Q. 
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f’3  u#in«  «»MUnt " O'  defined  la  B. 4. 3,  * general  expression  for 

th#  wil|hti  la  Caaaa  II  aad  IH  ou  bt  found  bv  uilnc  the  iAiAB  a#  n  o 


be  found  by  ualng  the  aaine  method  of  C,  3. 


(C  IU) 


uj  I  * 

j  J 
gl  3 


V_(CV 


J  -  J 
3  rf3 


J  Je 

.I  * 

where  D  «■  £ 


A 


Jj  •  0  •  o 


D  - 

^1  ft  "  J,  JL  J  -  J 

P0  qo  <°V  <l-CP0>*  8 


0  0  Ji  J2 

po<°V  qo  ^-Cp/ 


and  where  ^  -  4,  ^-1.0*2  for  Case  n,  C  »  10  for  Case  ni, 

and  the  term  ^0  ^0  ^  p®  (Cp^  Q0 1  (1  -  C  pQ )  3  in  the  denominator 
represents  the  state  •  0,  J2  »  o,  which  wtU  not  exist  in  screening. 

The  denominator  in  (C  III)  above  will  simplify  io 

>-<C  (‘-CP,,  ). 


"3 

1 

1 
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fetismmeeialSMMtte 


Then  from  (C  III), 


w  -  <1-CV4*>oqo 
10  - D - 


CPq*o 

D 


w*o"  <l-cPo)6pe  S>  wll“Cpo  4po*o 


(1  -  C  P  )  4  p  *  qrt  C  p  6  p*  q2 

O  O  O  w  m  o  °  0 


(I-Cpwp  ) 

w  ■  o  y 

«  — — 


w  .Cpo‘P(.  "» 

31  * - - - 

D 


wt.«CPoPo 
41  D 

where  D  «  1  -  qo4  (  1  -  C  pQ)  . 

C.  4  Note  that  (C  III)  reduces  to  (C  II)  when  J  «=  0.  Then  (C  III) 

2 

may  be  considered  the  general  expression  for  Case  I,  II  and  III. 


APPENDIX  D 


SPECIFIC  TO  GENERAL 


D.  1  Throughout  this  memorandum  tbs  word  "  specifier  hss  been  ussd 

ee  sa  adjsctive  to  modify  a  defect  whose  probability  of  ooourrSDoe  Is  different 
from  tbs  other,  equal  probability  of  occurrences,  This  restriction  was  made 
to  prevent  the  msthematloal  derivation  f-  >m  beoomlag  needlessly  lengthy. 

D.S  It  will  be  shown  here  that  this  restriction  Is  not  needed,  that  the 

results  will  be  the  same  if  the  particular  defect  having  the  odd  probability  of 
occurrence  is  not  specified. 

D.  3  Assume  we  have  J  defects  in  the  class,  where  the  Jth  defect 

has  probability  p  (J)  of  being  the  odd  one,  where  the  aum  p  (1)  ♦  p  (3)  ♦  .  .  .  . 

♦  P  (J)  «•  1* 


D.  4  Any  result,  R,  in  the  memorandum,  would  have  to  be  made 

subject  to  the  condition  that  the  1th  defect  was  the  odd  one.  However,  It  la 
Immediately  apparent  that  (R j  1)  ■  (R|  2)  -  .  .  .  -  (R|  j)  -  R,  since  the  given 
condition  would  be  equivalent  to  specification. 

D.  5  It  oan  therefore  be  stated  that  p  (1)  (R  |  1)  ♦  p  (2)  (R 1 2)  .  .  .  . 

*  P  0)  <R|  J)  -  (P  (1)  ♦  P  («)  ♦  •  .  .  .  +  P  (J)  )  R  »  R. 
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WPIRICAL  BAYES  ESTIMATION  FOR  THE 
APPLIED  STATISTICIAN 


Richard  Rrutchkoff 

Virginia  Polytechnic  Xoatltuta  and  State  University 

In  the  past  (aw  year#  «y  students  and  I  have  published  several  artlclaa 
(aaa  the  rafarancaa)  illustrating  tha  practicality  of  the  tout'd  Rayaa 
approach  to  atatlatlcal  estimation,  Unfortunatoly,  thaaa  .midst  wu  written 
in  a  way  which  doaa  not  easily  allow  for  extension  to  other  M^licatler'.  I 
am,  therefore,  writing  thla  non- technical  paper  eo  that  Empirical  Bayne  aatlaatlon 
can  ba  both  understood  aud  uaed  without  one  having  to  be  an  expert  In  the  field. 


When  to  Uee  tenirlcal  Rayaa 

i-»t  us  eay  that  you  want  to  eatlaate  the  parameter  6  (or  a  vector  of  para- 
ae’ert  f)  trow  a owe  specified  distribution.  You  take  your  staple  and  obtain  the 

base  classical  eatlaate,  $,  for  6.  Thla  le  usually  sufficient  for  your  purposes. 
If,  howrer,  you  feel  that  thla  eatlaate  la  not  good  enough,  perhaps  due  to 
large  experimental  error  or  eaall  eaaple  alto,  then  you  should  consider  other 
Alternatives. 

Let  ua  say  at  thla  point  you  realise  that  you  have  dona  all  thla  aavaral 
times  before.  That  la,  you  have  considered  alwllar  situations  and  have  estimated 
the  parameter  6  from  tha  same  distributional  fora.  The  first  queatlon  you  should 
aak  yourself  Is,  "ere  tha  prevtous  values  of  the  eetlaeted  parameter  the  earns  ee 
the  parameter  presently  being  estimated".  If  the  anewer  ta  this  Is  "yss",  than 
all  tha  data  should  be  pooled.  This  le  usually  dona  by  weighting  the  estimates 
proportional  to  the  number  of  observations  contained  therein.  If  the  variation 
of  f  between  experiments  Is  less  than  the  variance  of  your  unpooled  estimator, 
then  yoc  do  improve  your  estimate  in  this  way.  If,  on  the  other  hand,  the 
variation  of  6  between  experiments  le  larger  than  the  variance  of  your  unpooled 
estimator,  then  this  poolad  approach  affords  an  estimator  with  largar  mean 
squared  error  than  tha  unpooled  estimator.  What  Is  needed  in  this  cast  la  a 
walghced  avaraga  In  which  data  from  distributions  with  parameters  close  to  tha 
praaant  parameter  value  recolve  more  weight  than  date  from  distributions  with 
parameters  not  so  close.  A  natural  measure  of  closeness  la  provided  by  the 
probability  density  or  mess  function  itself.  The  estimators  obtained  In  this 
manner  ere  celled  ftspiricel  tayee  estimators.  They  hav#  been  derived  In  the 
literature  from  another  point  of  view  (see  the  references),  to  exhibit  optimal 
asymptotic  properties  end  have  been  shown  by  extensive  stochastic  simulation 
experiments  to  exhibit  excellent  smell  temple  properties.  In  particular  these 
estimators  ara  never  worse  than  tha  classical  set  las  tors  and  of tan  provldt  up 
to  a  50  percent  dscreasa  in  maan  squared  error. 
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There  1*  a  good  deal  of  evidence  which  indicataa  chat  the  Empirical  Beyea 
estimators  ara  robuac  to  moat  avarything.  If  the  previoua  exparlments  contain 
little  evidence  about  thia  experiment,  then  the  Empirical  Bayaa  aatlmata  dlffera 
little  from  the  claaalcal  eatimata  with  correapondlngly  little  dacreaaa  in  mean 
equared  error.  On  the  other  hand,  the  more  closely  the  expertmenta  leaembln  the 
present  one  tha  greater  the  decrease  In  mean  aquared  error. 

What  Eatimator  to  Use 


Although  I  have,  in  the  literature,  proposed  different  Empirical  Bayaa 
estimators  for  different  situations,  1  will  here  propose  only  one  easily  used 
procedure  for  any  parametric  situation.  Lot  us  say  that  f <x )  6) represent*  the 
probability  density  function  (or  tha  probability  mass  function)  of  the  data  for 
a  given  value  of  tha  parameter.  If  there  la  a  sufficient  statistic  for  e,  then 
let  g  represent  tha  sufficient  statistic  itself  rather  than  the  vector  of  obser¬ 
vations;  and  l*t,f (x|9J  represent  thv  trol  blllty  density  of  that  sufficient 
statistic.  Let  9  be  the  claaalcal  ea'.ia^  -r  cor  the  present  balue  of  6.  Now 
consider  the  last  several  past  experiince.*  (several  being  anywhere  from  five  to 
fifteen;  fifteen  whan  available).  Ust  6  ,  1  •  1,  2.  ...  ,  16.  to  represent  the 

classical  estimate*  in  the  past  fifteen  (or  aa  few  as  five)  experiences  plus  the 
present  one  (1  ■  16).  The  Empirical  *>/aa  estimate  for  the  present  value  of  6 
is  given  by 

SjfOcI^) 


f(xjei) 

Thle  le  e  weighted  average  of  the  estimate!  of  tfce  last  sixteen  a  values.  The 
weights,  however,  are  not  functions  of  how  well  6  is  estimating  0^,  1.*,,  a 

function  of  the  number  of  observations  in  the  1th  experiment,  but  rather  a  function 

of  how  llkaly  the  preaent  set  of  data,  x,  would  ba  If  0.  were  the  true  value  of 

*  1 

the  parameter.  It  should  be  noted  that  the  present  unpooled  estimate  9  ^  recleves 
the  greatest  weight.  This  Is  more  easily  ssen  when  x  -  9^  and  the  weight  Is 
proportional  to  f(P  16l916''* 

The  MbA  ot  the  toplrlcal  Bayes  Estimator 

The  decreaae  in  mean  squared  error  using  this  Empirical  Bayes  procedure 
depends  on  Che  ratio  of  the  variance  of  the  unpooled  eatimator  to  the  variation 
of  the  parameter  between  experiment*.  Unlike  ordinary  pooling,  howevsr,  the 
Empirical  Bayes  estimators  never  heve  a  larger  mean  squared  error  than  the 
unpooled  eatimator. 


16 

Z 

1-1 


16 

1-1 
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Sine* 


Vn  i  •  E  v«r(e|e)  ♦  Var  B(e|s) 


we  have 


Var  9  »  Var  §  -  E  Var(8  |9) 

A 

whan  9  la  unbiased  for  8.  We  can,  therefore,  ofcan  praaant 

8 .  v*nitei _ _ 

Var  6  -  B  Var(e|e) 

aa  tha  ratio  required  for  uae  with  labia  1  In  obtaining  the  (aproxlmate  at an 
squared  arror  for  your  Empirical  Bayes  estimator.  Hera  Var(9|6)  la  estimated 
aa  tha.varlanco  of  tha  unpooled  estimator  for  your  present  experiment. 

E  Var (0 | 9)  Is  estimated  by  the  average  of  the  variances  of  tha  unpgoled 
estimators  for  the  present  and  fifteen  past  experiments,  while  Var  6  Is 
estimated  by 


1 

T* 


16 

r 

1-1 


1  16  -  2 

16  1  V 

16  J-l  j 


table  I  la  an  approximate  table  which  has  been  compiled  by  using  extensive 
stochastic  simulations.  One  finds  the  Z  value  and  obtains  the  proper  multiplier, 
Interpolating  when  necessary.  The  variance  of  the  unpooled  estimator  la  then 
multiplied  by  this  multiplier  to  obtain  the  mean  squared  error  for  the  Empirical 
Bayes  estimator.  If  Z  is  very  small,  under  0.1,  then  there  is  little  advantage 
In  obtaining  the  Baplrical  Bayes  estimator.  .f,  on  the  other  hand,  Z  la  very 
large,  over  4.0,  then  one  should  seriously  consider  pooling  In  the  usual 
manner.  For  0.1  Z  _<  4.0  one  should  use  the  estimator  given  here. 

When  Z  Is  baaed  on  only  five  past  experiences,  the  multipliers  art  closer 
to  unity.  A  reasonable  estimate  would  be  to  split  the  difference  between  them 
and  unity,  e.g,,  a  multiplier  of  0.8  would  become  0.9  when  only  5  past  experiences 
are  available. 


Table  I»  Variance  Multipliers 


Multiplier 


0.0 

0.2 

0.5 

1.0 

2.5 

5.0 


1.0 

0.9 

0.8 

0.7 

0.6 

0.5 
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Sometimes  we  feel  that  our  data  la  almost,  but  not  quite  normal  or  not 
quite  polsson  or  not  quite  gamma,  etc.  In  spite  of  this,  we  often  make  the 
assumption  that  our  data  fits  close  enoug  In  order  to  avoid  havlna  to  use  a 
nonpar ami' trie  procedure,  which  la  Itself  very  powerful,  the  nonparametrlc 
Empirical  Bayes  procedure  which  1  am  prei  ng  here  is,  however,  remarkably 
powerful  and  competes  well  with  its  paramo  c  counterparts. 


Let  x.,  1*1,  2,  ...  ,16,  represent  l  lest  sixteen  ?ctor  observations, 
including  the  present  one  with  1  -  16.  Thai  .e,  5  ,  1*1,  16, 

represents  the  sets  of  random  samples  obtains-  In  the  last  s  xteen  experimental 
situations  with  the  present  set  of  random  samples.  Let  ,  J  •  1,  2 . m^, 

represenr  the  components  of  the  ith  vector.  Note  that  the  vector  lengths  need 
not  be  the  same  but  m^  should  he  at  least  two. 

Let  us  now  say  that  we  have  a  classical  estimator  for  whatever  It  Is  we 
want  to  estimate.  In  the  nonparametric  situation  this  would  be  something  like 
the  mean,  the  variance,  the  standard  deviation,  or  the  percentiles. 


Call  the  estimates  in  the  last  sixteen  situations  0  ,  i  •  1,  2,  . ..,16, 

where  6  Is  tne  present  classical  estimate  of  what  we  are  interested  in 
xo 

estimating.  Then  the  Empirical  Bayes  estimator  is  given  by 

16  *  , 
e  e1f(xl6;ji) 

5  -  it - 

f(«16;5i) 


where 


with 
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and  xt  -  (1/m^ 


1 

Z 

k-1 


x^  and  whare 


■  In  0 


-  1. 


Tha  f(;16u1) 


given  hara  ta  simply  a  nonpersmetrlc  estimate  of  Cha  appropriate  likelihood 
function.  Preliminary  results  indicate  that  this  procedure  ia  also  remarkably 
robuat.  In  particular,  tha  lengths  of  the  vectors  from  experiment  to  experiment 
need  not  be  the  seme. 
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A  STATISTICAL  APPROACH  TO  OPTIMIZING  THE  MECHANICAL 
BEHAVIOR  OF  COMPOSITE  MATERIALS 


Donald  L.  Martin,  Jr.* 


ABSTRACT 


The  filler  particle  size  distribution  is  shown  to  have  a  significant  effect  on 
the  mechanical  properties  of  CTPB  composite  materials.  The  experimental  com¬ 
positions  are  selected  in  accordance  with  a  simplex-centroid  lattice  design,  and 
the  regression  equation  for  this  statistical  model  is  shown  to  adequately  fit  the 
experimental  data.  The  various  mechanical  properties  estimated  from  the 
regression  equation  for  a  fixed  filler  content  are  represented  by  contour  plots  on 
triangular  coordinate  graphs.  These  plots  predict  the  various  possible  mixtures 
that  will  result  in  the  same  material  behavior,  and  should  prove  valuable  in  optimiz¬ 
ing  mechanical  properties  of  composite  materials  for  specific  applications. 


INTRODUCTION 


Composite  materials  are  currently  being  used  in  many  applications  where 
the  strength-to-weight  ratio  becomes  a  significant  factor.  The  extreme  loading 
conditions  imposed  on  these  materials  has  prompted  many  investigations  of 
methods  for  optimizing  the  mechanical  properties  for  specific  designs. 

The  work  described  here  concerned  a  carboxyl-terminated  polybutadiene 
(CTPB)  composite  material  that  consists  of  a  rubbery  matrix  highly  filled  with 
rigid  filler  particles.  The  size  distribution  of  these  filler  particles  and  the  total 
filler  content  have  a  pronounced  effect  on  the  rheological  characteristics  of  an 
uncured  slurry  (1-4),**  and  may  drastically  alter  the  mechanical  response  of  the 
cured  composite  material.  Such  characteristics  as  the  equilibrium  modulus, 
stress-strain  response,  and  ultimate  tensile  properties  [5-aj  of  a  cured  composite 
material  may  be  significantly  changed  by  altering  the  particle  size  distribution  for  a 
fixed  filler  content. 


♦Aerospace  Engineer,  U.  S.  Army  Missile  Command,  Redstone  Arsenal,  Alabama. 

**  Numbers  In  brackets  pertain  to  references  listed  at  the  end  of  this  paper. 

This  paper  has  baan  reproduced  photographically  from  the  author' a  manuscript. 
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Since  high  strength  oompositea  usually  require  a  high  solids  content,  the 
solids  must  be  efficiently  packed  to  obtain  optimum  properties.  The  packing 
fraction  as  defined  here  is  the  volume  fraction  of  solids  In  the  minimum  sediments* 
tlon  volume,  which  la  thus  Independent  of  the  actual  total  filler  content  In  the 
formulation.  When  the  particle  shape,  average  slse.  or  stse  distribution  are 
changed  such  that  the  packing  fraction  is  increased,  leas  binder  is  required  to  Oil 
the  voids  between  the  filler  particles.  Thus  there  Is  a  direct  relationship  between 
the  packing  fraction  of  the  mixture  of  Oiler  particles  and  the  potential  solids* 
loading  capability  of  the  resulting  composite.  For  a  fixed  filler  oontent,  the 
largest  packing  fraction  usually  results  in  the  minimum  mix  viscosity,  the  lowest 
initial  modulus,  and  the  greatest  strain  capability. 


Optimising  a  mixture  of  filler  particle  sices  from  the  standpoint  of 
rheological,  physical,  and  thermal  properties  requires  knowledge  of  the  mechani¬ 
cal  response  of  all  possible  mixtures  that  might  be  considered.  Since  the 
experimental  determination  of  qll  the  various  responees  is  prohibitively  expensive 
and  time-consuming,  a  method  for  accurate  prediction  is  highly  desirable. 


i 


BACKGROUND  AND  THEORY 


The  ooooept  of  simplex  lattice  designs  Introduced  by  Scheffd  [9]  for 
experimental  situations  involving  multicomponent  systems  Is  valid  when  the 
experimental  response  depends  only  on  the  proportion  of  components  in  the 
system.  To  Illustrate,  let  q  be  the  number  of  components  considered  and  X 

th  * 

the  proportion  of  the  1  component  that  la  subject  to  the  constraints 

Xj6  0  (1  -  1,  2,  ...  q)  (1) 


and 


X  +  X„  +  X  ♦ 
12  3 


+  X  ■  1 

q 


(2) 


The  factor  space  Is  a  (q-1)  -dimensional  simplex  satisfying  these  constraints 
(triangle  for  q  ■  3,  tetrahedron  for  q  3  4) .  Scheffd  uses  the  nomenclature 
{q,m}  to  describe  tbe  designs,  where  m  is  the  degree  of  the  polynomial 
regression  function  In  X{  used  to  describe  components.  The  {q.m}  simplex 

lattice  (m  »  1, 2, 3, ... )  consists  of  the  experimental  points  In  the 

simplex  representing  all  possible  mixtures  In  which  the  proportion  of  each 
component  has  the  m+1  equally  spaced  values: 


X  =  0  —  — 
1  mm 


1 . 


(3) 
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The  simplex  lattice  method  has  two  key  features:  (1)  Properties  or  responses  are 
measured  at  lattice  composition  points  and  (2)  polynomial  equations  having  a 
special  oorrespondenoe  to  the  lattice  points  are  then  used  to  represent  the 
measured  responses  at  these  points. 


The  {3,2}  simplex-centroid  lattice  design  proposed  by  Scheffd  (10)  was 
satisfactorily  applied  by  Alley  and  Dykes  (11)  to  estimate  the  packing  fractions  of 
multi  component  mixtures  of  ammonium  perchlorate  and  aluminum  particles. 

In  the  simplex-centroid  statistical  model,  (2^-1)  observations  are  taken, 
one  at  each  of  the  following:  the  q  pure  components,  the  (q/2)  binary  mixtures 
with  equal  proportions,  the  (q/S)  ternary  mixtures  with  equal  proportions,  .... 
and  the  q-nary  mixture  with  equal  proportions.  These  observations  correspond 
to  points  X^,  Xg,  ....  Xq  of  the  simplex,  obtained  by  making  q  permutations  of 

<1,0,0 . 0),  (q/2)  permutations  of  <1/2,  1/2,  0,  ....  0),  and  <q/3) 

permutations  of  (1/3,  1/3,  1/3,  0,  ....  0),  and  the  point  (l/q,  1/q . 1/q). 

Thus  the  design  consists  of  the  centroid  of  the  (q,2)  simplex  and  the  centroids  of 
all  the  lower  dimensional  aimplexes  It  contains. 

The  polynomial  regression  equation  has  as  many  coefficients  as  there  are 
points  In  the  design,  and  is  given  by 

q  q-1  q 


1  “  l  ^1  *  Z  l  /Wl 

1=1  i=l  j=,i+i  1 


q-2  q-1  q 

III  Vxixixk 

1=1  H+l  k=j+l  3  3 


+  *12...qXlX2-Xq 


q 

When  the  response  satisfies  (4),  the  { 2  —1)  coefficients  0  are  uniquely  deter¬ 
mined  by  the  values  of  the  response  at  the  (2^-1)  points  of  the  simplex-centroid 
design  1 10):  The  coefficient  0^  in  (4)  is  the  response  of  the  pure  component  i, 

called  the  linear  blending  value;  0^  is  the  coefficient  of  binary  synergism  for 

components  i  and  );  0  Is  the  coefficient  of  ternary  synergism  for  components 
ljk 

1,  ],  and  k;  etc. 


Rsfsrsnoe  Mixtures 


In  many  experiment*  dealing  with  mixtures,  a  physical,  economic,  or 
chemical  limitation  may  restrict  the  concentration  of  one  or  more  components 
of  s  mixture.  For  these  oases  Scheffd  suggested  the  use  of  pseudocomponents,  or 

what  were  later  termed 
"reference  mixtures," 
which  are  mixtures  of  the 
pure  oompoaenta  and  can 
be  substituted  for  the 
pure  components  In  the 
simplex  lattice  designs. 
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Figure  1.  GRAPHICAL  REPRESENTATION  OF  A 
{3,2}  SIMPLEX-CENTROID  LATTICE 
DESIGN  FOR  PURE  COMPONENTS 
AND  REFERENCE  MIXTURES 


Womeldorph  (12) 
and  Alley  and  Meyers  (13) 
incorporated  reference 
mixtures  Into  the  simplex- 
centroid  design  for  multi- 
component  x-ray  analysis. 
Figure  1  Illustrates  two 
{3,2}  simplex-centroid 
designs,  one  for  mixtures 
of  pure  components 
(designated  by  I)  and  the 
other  for  reference 
mixtures  (II).  Point  A 
is  a  mixture  of  pure  com¬ 
ponents,  but  this  mixture 
is  also  100  percent  refer¬ 
ence  mixture  1.  By  a 
similar  definition  for 
using  reference  mixtures 


points  B  and  C,  a  {3,2}  simplex-centroid  design  (II) 

Is  obtained.  The  regression  equation  In  terms  of  the  reference  components  is 
similar  to  that  for  the  pure  components.  Although  in  this  particular  illustration 
the  reference  component  and  pure  component  designs  have  the  same  midpoint, 
this  condition  is  not  required. 


Variance  and  Confidence  LlmltB 

To  verify  the  adequacy  of  the  model  and  to  establish  confidence  intervals, 
the  variance  of  the  values  predicted  by  the  model  must  be  estimated.  Variances 
dealing  with  any  number  of  components  are  discussed  in  detail  by  Scheffd  (9)  and 
Womeldorph  (12)  and  by  Gorhman  and  Hinman  (14),  who  extended  the  simplex 
lattice  design  and  applied  it  to  properties  of  gasolines  and  other  multicomponent 
systems.  The  discussion  here  will  concern  the  expression  for  the  variance  of 
values  nredicted  from  the  {3,2}  simplex-centroid  model. 


The  coefficients  0,  0  ,  and  0  0  In  the  {3,2}  at mplex-eentr old  model, 

1  lj  1*0 

3  2  3 

” '  £,  &  Vixi  *  "mWa  •  151 


are  estimated  from  linear  combinations  of  the  measured  response  at  the  lattice 
points  [0].  Replacing  the  0’s  by  their  estimates  in  terms  of  the  measured 
responses  v  yields 


3 

n  *  £  bi  + 

i=i 


2  3 

l  l 

i-1  H 


‘ViJ 


b123ni23 


(6) 


If  an  Independent  estimate  of  the  error  variance,  a7,  is  not  available,  thi  design 
must  be  replicated  to  obtain  an  estimate  of  a2: 


(7) 


where  92  is  the  averafte  variance  for  the  model,  <r2  is  the  variance  of  the  1th  mixture, 
and  p  is  the  number  of  distinct  design  points  in  the  model  considered. 


The  Xj's  are  assumed  to  be  known  without  error  and  the  rj's  result  from 

replicated  observations  at  each  lattice  point.  The  expected  variance  in  the 
predicted  response  is  then  given  by 


Var  (71)  =  o2 


+ 


(8) 


When  there  are  an  equal  number  of  observations,  r,  at  each  lattice  point 
composition. 


and 


« 


123 


Var  (?J) 


(») 
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whare 


3  2  3 

■  Z  *>\  ♦  Z  Z  v  *  b> 

b»l  1  1-1  J-l+1  **  183 

(10) 

3 

(*,/*)(«!  -«!*>)-»  £  x! 

(11) 

1-1 

\  ■  «,x,  (»i  ♦»)-’) 

(12) 

bl!3'27XlX2V 

(13) 

The  quantity  <r*/r  ia  dependent  on  the  precision  of  the  experimental  observations  and 
C  is  dependent  only  on  the  composition  being  considered.  Thus,  if  an  estimate  is 
given  of  cr*  and  the  number  of  replications  r  for  the  simplex-centroid  system, 
the  estimated  variance  fot,  any  composition  in  the  factor  space  can  be  obtained  by 
using  (9)  through  (13).  For  95  percent  confidence  Intervals  [  ll) : 

Pr{fT-A<*)<?j+A)  *  0.95  ,  (14) 


where 


‘■Wi'Ki,r 

a  -m  0. 05 

k  ■  number  of  constants  in  model 
f  -  number  of  degrees  of  freedom. 

When  desirable,  a  contour  plot  of  confidence  intervals  for  the  predicted  values 
may  be  prepared  by  using  (14) . 


EXPERIMENTAL  PROCEDURE 


Material 

The  composite  system  selected  for  this  investigation  consisted  of  a  CTPB 
binder  filled  with  rigid  filler  particles.  The  filler  material  was  obtained  from 
Rohm  fit  Haas  Company,  Redstone  Arsenal,  Alabama.  Table  I  presents  the 
nominal  size,  weight-mean  diameter,  and  specific  surface  area  for  the  filler 
particles.  In  the  ensuing  discussion,  the  nominal  size  will  be  used  to  identify  the 
filler  component.  The  particle  size  distribution  of  the  200  n  particles  was 
determined  by  Ro-tap  screen  analysis;  the  particle  size  distributions  of  the  40  ^ 
and  10  n  particles  were  determined  by  a  liquid  sedimentation  technique  using  a 
Mine  Safety  Appliances  analyzer. 
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Table  1.  PARTICLE  SIZE  DATA  FOR  FILLER  PARTICLES 


MATERIAL 

NO. 

NOMINAL 

SIZE 

M 

WEIGHT-MEAN 

DIAMETER 

<F> 

SPECIFIC 
SURFACE  AREA 
(CM*/G> 

1 

40 

44.6 

1440 

2 

200 

181.7 

100 

3 

10 

16.6 

1080 

The  cumulative  particle  else  distributions  are  plotted  in  Figure  2.  The 
200  ji  particle  sites  ranged  from  74  n  to  380  p  and  had  a  weight-mean  diameter 
of  191. 7  p.  The  40  p  particle  slaea  ranged  from  3  p  to  80  p,  with  a  weight-mean 
diameter  of  44. 5  p.  The  10  p  particle  slaes  ranged  from  3  n'to  30  p  with  a 
weight-mean  diameter  of  16. 6  p.  Calculation  of  the  specific  surface  areas  and 
the  weight-mean  diameters  was  based  on  particle  else  distribution  data  and 
assumed  spherical  particles. 


Figure  2.  PARTICLE  SIZE  DISTRIBUTION  CURVES 
FOR  FILLER  PARTICLES 


The  raw 

materials  used  In 
the  binder  formula¬ 
tion  of  all  composi¬ 
tions  Investigated 
were  obtained  from 
Thlokol  Chemical 
Corporation,  Hunts¬ 
ville,  Alabama.  The 
chemical  analyses  of 
the  CTPB  prepolymer 
and  curing  agents 
are  listed  In  Table  H. 
The  CTPB  system  uses 
the  carboxyl  groups  at 
the  terminal  positions 
and  the  butadiene 
polymer  as  the  back¬ 
bone.  The  ERLA  is 
a  trifunctional  epoxide 
compound  and  the 
MAPO  is  a  trifunctional 
imine  compound. 
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Table  II.  RAW  MATERIALS 


POLYMER  LOT  NO.  EQUIVALENT 

WEIGHT 

CTP0-ZL434  3611  2080.0 

PR  LA -06 10  2S27  M.86 

MAPO  2506  73.63 

The  equivalence  ratio  of  the  binder  component  In  all  compositions  Investi¬ 
gated  was  maintained  constant  at  1. 5  equivalents  of  curing  agents  to  one  equivalent 
of  CTPB  prepolymer.  Hie  ratio  of  curing  agents  vas  maintained  constant  at  3.  8 
equivalents  of  MAPO  per  one  equivalent  of  ERLA.  The  composite  formula  tluna 
investigated  are  summarised  in  Table  III. 


Table  III.  COMPOSITE  FORMULATIONS 


VOLUME 

PERCENTAGE 

MIX  NO. 

BINDER 

40 u 

200 p 

10m 

314 

0.40 

0.80 

0.0 

0.0 

311 

0.40 

0.0 

0.60 

0.0 

317 

0.40 

0.0 

0.0 

0.60 

318 

0.40 

0.30 

0.30 

0.0 

319 

0.40 

0.36 

00 

0.26 

320 

0.40 

0.0 

0.35 

0.25 

321 

0.40 

0.22 

0.22 

0.16 

Mechanical  Rest 


Uniaxial  stress-strain  characteristics  were  determined  utilizing  nn 
Instron  tensile  machine  at  a  corutant  crosshead  displacement  .  ate  r>f 
1.0  lncl/ minute  and  7S*F.  Borded  tab-cna  specimens  approximately  3/8  inch 
wide,  1/4  Inch  thick,  and  2.  8  Inches  '.onjr  were  used  to  obtain  the  mechanical 
response  data.  Three  specimen*  <r.  each  composition  were  tested  and  the 
average  value  for  each  property  was  used  in  the  subsequent  correlation. 
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Crosslink  Density 

The  mechanical  properties  of  polymeric  binders  may  change  with  the 
addition  of  rigid  filler  particles  due  to  the  reinforcing  effect  of  filler  particles, 
changes  produced  In  the  crosslink  density  of  the  binder,  or  a  combination  of 
the  two.  For  some  binder  systems  the  addition  of  particular  filler  materials 
results  in  an  Increase  In  the  apparent  crosslink  density  of  the  binder,  while  other 
filler  materials  hinder  the  cure  reaction  and  cause  a  decrease  In  the  apparent 
crosslink  density.  Therefore,  to  adequately  account  for  ln£*tence  of  filler 
materials  on  a  polymerio  binder,  It  la  necessary  to  determine  Jie  relative 
effects  of  changes  In  the  filler  p&rtlcloa  and  crosslink  density. 


The  crosslink  density  of  each  lattice  point  composition  was  determined 
from  compression  and  tensile  properties  at  equilibrium  swelling  conditions  In 
benzene.  The  number  of  moles  of  effective  crosslinks  per  unit  volume, 

for  elastomeric  baaed  materials  was  then  calculated  from  the  theory  of  rubber 
elasticity  which  predicts  ( 15  ] 


F_ 

A, 


RT  (X  -  X“l)  , 


where 


F  -  force  (dynes) 

Aj  »  the  initial  cross-sectional  area  of  the  unstrained,  unswollen 
specimen  (cm1) 

R  m  universal  gas  constant,  8.314  x  10  dynes/ mole  (*K) 

T  -  test  temperature  (*K) 

X  »  extension  ratio. 


Packing  Fraction 


The  packing  fraction  or  minimum  sedimentation  volume  of  filler  particles 
was  determined  according  to  the  following  procedure:  Samples  of  the  filler 
material  of  each  of  the  lattice  point  compositions  were  placed  in  15-miililiter 
centrifuge  tubes  and  the  weight  of  each  sample  was  accurately  determined.  Each 
mixture  sample  was  blended  with  a  Pica  blender  and  enough  Twitchell  base 
n-heptane  was  added  to  completely  cover  the  material.  After  the  slurry  was 
stirred  with  a  microspatula  to  completely  wet  the  particles  and  dispel  air 
bubbles,  It  was  centrifuged  at  2900  rpm  and  the  volume  of  the  solids  measured 
at  20-mtnute  Intervals  until  a  constant  volume  was  obtained.  In  most  cases  this 
determination  required  60  minutes  or  less,  with  the  samples  containing  the 
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largest  proportion  of  smaller  particle  sties  requiring  the  longest  time.  The 

packing  fraction,  $  ,  of  the  sample  was  then  determined  bv  the  relationship 
m 

m  p 

where  P  is  the  bulk  density  of  the  sample  at  the  minimum  sedimentation  volume 

and  p  is  the  theoretical  density  of  filler  particle  mixture.  Thus  the  packing 

fraction  is  the  fTaotlon  of  the  minimum  sedimentation  volume  that  is  actually 

occupied  by  the  filler  particles,  and  1  -  0  would  represent  the  minimum  amount 

m 

of  binder  that  would  theoretically  be  required  to  completely  fill  the  voids  between 
the  filler  particles. 


RESULTS 


The  results  of  the  investigation  indicated  that  the  mechanical  properties 
of  the  composites  considered  are  strongly  Influenced  by  the  particle  Blze  distri¬ 
bution  of  the  filler  material.  Listed  in  Table  IV  are  the  experimentally  deter¬ 
mined  packing  fraction,  <p  ;  maximum  corrected  stress,  a  *;  strain  at  maximum 
0  m  m 

corrected  stress,  c  ;  maximum  initial  tangent  modulus,  E;  crosslink  density, 
m  ” 

and  normalized  tangent  modulus,  E/*^. 


Table  IV.  MEASURED  RESPONSES  OF  LATTICE  POINT  COMPOSITIONS 


MIX 

NO. 

°m 

(PSD 

em  X  10* 
(IN/IN) 

E 

(PSD 

v,  X  106 
(MOLES/CM3) 

E/v.  X  10-® 
(PSI/MOLE  CM'3) 

314 

0.730 

147.4 

8.86 

1780 

19.9 

8.96 

311 

0.680 

100.7 

8.86 

1510 

11.2 

13.50 

317 

0.736 

260.5 

15.1 

1990 

44.6 

4.46 

318 

0.778 

114.7 

8.0 

2080 

15.8 

13.20 

318 

0.754 

174.3 

12.0 

1580 

25.0 

6.30 

320 

0.834 

118.2 

11.7 

1545 

13.9 

11.10 

321 

0.804 

132.7 

11.7 

1410 

7.2 

7.22 

*The  maximum  corrected  stress  a  as  used  in  this  investigation  is  defined 

in 

as  the  nominal  stress  multiplied  bv  the  extension  ratio. 


120 


From  these  values  the  coefficients  fp  , 
presented  In  Table  V.  '  1 


were  determined  and  are 


Table  V.  COEFFICIENTS  OF  REGRESSION  POLYNOMIAL 


COEFFICIENT 

°m 

em 

E 

E/r. 

*1 

0.730 

147.4 

8.88 

1780 

19.9 

8.96 

? 

02 

0.880 

100.7 

8.88 

1610 

11.2 

1360 

| 

03 

3.738 

260.5 

16.1 

1090 

44.6 

4.48 

i 

012 

0.292 

37.4 

•344 

1740 

1.0 

7.90 

i 

013 

0.089 

-87.8 

1.97 

•1184 

21.94 

•2.98 

023 

0.S33 

•207.9 

0.691 

-866 

-47.36 

6.00 

0123 

0.263 

174.8 

38.76 

•10639 

263.1 

•99.50 

- 

«u 

\ 


Figure  3.  PACKING  FRACTION  VERSUS  FILLER 
PARTICLE  SIZE  DISTRIBUTION 


The  numerical 
coefficients  were  sub¬ 
stituted  Into  the 
regression  equation  «  4 > 
to  predict  the  various 
compositions  that  will 
exhibit  the  same  mech¬ 
anical  behavior  character¬ 
istics.  This  information 
is  displayed  on  triangular 
coordincto  graphs  as 
contour  plots  of  constant 
response  with  respect 
to  the  individual 
properties  Investigated. 

The  first  of  these 
plots  (Figure  3)  indicates 
the  contours  of  constant 
packing  fraction,  which 
is  dependent  only  on 
the  proportion  of  the 
various  particle  sizes. 
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Figure  4.  MAXIMUM  CORRECTED  STRESS  AS 
INFLUENCED  BY  FILLER  PARTICLE 
SIZE  DISTRIBUTION  Ax  ,  pai\ 


The  next  three 
graphs  (Figures  4,  5, 
and  6)  show  the  effect 
of  filler  particle  size 
distribution  on  the  maxi¬ 
mum  corrected  stress, 
strain  at  maximum 
corrected  stress,  and 
maximum  initial  tangent 
modulus,  respectively. 

In  all  compositions  the 
filler  content  was  main- 
rained  constant  at  60 
P'rcent  by  volume.  Hie 
coordinates  of  the  graphs 
represent  the  proportions 
of  40  p ,  200  p  ,  and  10  p 
particle  sizes  in  a  given 
mixture;  the  vertices 
of  the  graphs  represent 
the  composite  formula¬ 
tions  in  which  the  total 
filler  content  consisted 
of  100  percent  of  the 
nominal  particle  size 
designated. 


40  *4 


Figure  5.  STRAIN  CAPABILITY  OK  CTPB  COMPOSITE 
MATERIALS  VERSUS  FILLER  PARTICLE 
SIZE  DISTRIBUTION  (<m,  in. /in.  x  10-) 
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The  maximum 
corrected  stress  (Fig¬ 
ure  4)  varies  from  100 
to  260  psi,  depending  on 
the  fl’ler  particle  size 
distribution,  and  in 
general  increases  with 
increasing  content  of 
40  p  and  10  p  filler  par  ¬ 
ticles,  with  the  largest 
stress  value  occurring 
near  the  100  percent 
10  p  point. 

The  strain  at 
maximum  corrected 
stress  (Figure  5)  varies 
from  0.  08  to  0. 151  inch 
per  inch  depending  on  the 
filler  particle  distribu¬ 
tion,  and  increases  with 
the  increase  in  10  p 
•  ■ontent. 


uw 


Figure  6.  MAXIMUM  INITIAL  TANGENT  MODULUS 
VERSUS  FILLER  PARTICLE  SIZE 
DISTRIBUTION  (E  x  1(TS,  pfli) 


*o  *» 


Figure  7.  APPARENT  CROSSLINK  DENSITY  VERSUS 
FILLER  PARTICLE  SIZE  DISTRIBUTION 
(l’e  x  io5,  moles/cm5) 
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The  maximum 
initial  tangent  modulus 
(Figure  6)  varies  from 
1400  to  2080  pal, 
depending  on  particle 
size  distribution,  and 
reaches  a  minimum 
value  when  the  mixture 
of  filler  particles  con¬ 
sists  of  32  percent  200  n , 

28  percent  40  u,  and 
40  percent  10  u  particles. 

The  stress,  strain, 
and  modulus  contours 
bear  no  resemblance  to 
the  constant  packing  fraction 
contours,  indicating  that 
the -filler  material  has 
an  effect  on  the  binder 
cure  reaction.  To  deter¬ 
mine  the  extent  of  this 
effect,  the  apparent 
crosslink  density  was 
determined  for  each  lattice 
point  composition,  using 
the  procedure  described 
previously.  The  resulting 
values  appear  in  the  sixth 
column  of  Table  IV.  The 
corresponding  column  of 
Table  V  gives  the  coeffi¬ 
cients  determined  for  the 
regression  equation. 

Plotted  in  Figure  7  are 
the  estimated  contours  of 
constant  crosslink  density 
values  within  the  factor 
space.  These  contours 
are  very  similar  to  those 
of  the  initial  tangent  modu¬ 
lus,  with  the  exception 
that  the  point  where  the 
minimum  crosslink  value 
occurs  indicates  a  greater 
proportion  of  200  n 
particles. 


Figure  7  provides  further  evidence  that  the  filler  material  alters  the  cure 
of  the  binder:  the  crosslink  density  varies  from  7. 2  x  10“5  to  44. 6  x  io"s 
molea/cm5,  depending  on  the  particle  alee  distribution  used. 


As  was  indicated  earlier,  the  maximum  initial  tangent  modulus  for  cross- 
linked  polymeric  binders  varies  in  direct  proportion  to  the  crosslink  density  [IS], 

So  that  the  additional  step  of  normalising  the  modulus  by  the  crosslink  density 
was  necessary  to  clearly  reflect  the  reinforcing  effect  of  particle  size  distribution. 
The  normalised  values  of  E/r  for  each  lattice  point  composition  are  included  in 

Table  IV  (last  column) , 
and  the  corresponding 
column  of  Table  V  gives 


the  coefficients,  0, 
determined  for  the 
regression  polynomial. 
The  graphic  plot  of 
the  information 
(Figure  8)  represents 
contours  of  constant 
values  of  Initial  tangent 
modulus,  normalized  to 
unit  crosslink  density, 
as  a  function  of  the 
particle  size  distribution. 

The  normalized 
initial  tangent  modulus 
was  found  to  vary  from 
4.46  x  10e  to  13.5  x  106 
pal/ mole-cm"5,  depend¬ 
ing  on  the  particle  size 


distribution  used.  The 


Figure  8. 


TANGENT  MODULUS  NORMALIZED 
UNIT  CROSSLINK  DENSITY  VERSUS 


FILLER  PARTICLE  SIZE  DISTRIBUTION 


x  10”*,  psi/mole-cm”3  J 


value  was  lowest  at  the 
100  percent  10  ^  size 
and  increased  with  in¬ 
creasing  content  of  40  u 
and  200  v  sizes. 


CONCLUSIONS 


With  only  a  small  amount  of  experimental  data,  the  effect  of  filler  particle 
distribution  on  the  mechanical  properties  of  CTPB  composites  can  he  rapidly  and 
simply  approximated  by  simplex-centroid  experimental  designs.  The  most  signifi¬ 
cant  feature  of  this  lattice  design  is  the  case  with  which  the  regression  equation  can 
be  derived.  The  values  cf  the  coefficients  are  uniquely  determined  and  are  equivalent 
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to  the  estimates  that  would  be  obtained  by  a  standard  least  squares  analysis.  It 
should  be  noted  that  the  estimated  coefficient  values  are  not  changed  by  the  addition 
or  deletion  of  components  in  a  design. 

The  technique  la  aubjeot  to  some  risks  since  it  is  used  to  estimate  wide 
variations  in  unknown  compositions  with  a  limited  data  base;  however,  the  results 
obtained  in  this  Investigation  Indicated  that  the  mechanical  property-oompoeltlon 
contours  are  well-behaved  and  can  be  adequately  represented  by  the  simplex- 
centroid  model. 

Although  the  subject  system  contained  a  fixed  volume  fraction  of  binder, 
the  method  can  theoretically  be  extended  to  estimate  the  response  for  variations 
in  total  filler  content  by  using  the  binder  as  a  fourth  component.  In  addition,  a 
similar  design  can  be  devised  to  incorporate  other  component  siaes  of  filler 
particles  by  merely  adding  the  appropriate  coefficients  to  the  regression  equation. 

The  extension  of  this  method  to  more  than  three  components,  however, 
presented  certain  difficulties.  The  comments  and  suggestions  of  the  clinical 
session  panelists  are  particularly  solicited  with  respect  to  the  following  diffi¬ 
culties  encountered: 

(1)  When  investigating  composite  msterials  of  the  type  used  in  this 
study,  we  are  restricted  to  the  use  of  reference  components.  However,  the 
formulator  of  such  composite  materials  requires  the  Information  in  terms  of  the 
pure  components.  Therefore,  for  optimum  application  of  this  technique,  it 
becomes  necessary  to  transform  the  response  function  in  terms  of  reference 
mixtures  to  a  response  function  with  pure  component  concentrations  as  the  so- 
called  Independent  variables.  Womeldorph  1 12  J  presented  the  resulting  trans¬ 
formations  for  a  three-component  system.  The  same  reasoning  used  by 
Womeldorph  can  theoretically  be  applied  to  any  number  of  reference  mixture 
components  but  the  mathematical  complexity  of  the  transformation  Increases  by 
orders  of  magnitude  with  the  addition  of  extra  components  to  ihe  model.  The 
simplex-centroid  design  appears  less  attractive  In  these  cases  and  it  is  doubtful 
that  the  designs  would  be  widely  used  for  more  than  three  components.  To 
overcome  this  difficulty  it  is  planned  to  use  equation  (4)  and  express  the  com¬ 
posite  formulation  in  terms  of  the  pure  components  (X)  instead  of  reference 
mixture  components  (X*) .  The  (q,  2}  simplex-centroid  concept  would  thus 
result  in  a  set  of  q  simultaneous  equations  which  could  easily  be  solved  for  the 
unknown  coefficients  using  a  high  speed  computer.  The  system  of  experimental 
points  designated  In  Figure  1  was  utilized  as  a  check  on  this  technique.  Identical 
plots  were  obtained  for  systems  1  and  II.  The  actual  composition  of  the  points 
on  n  in  terms  of  the  pure  components  were  used  Instead  of  X*.  Will  this 

give  any  confidence  that  the  Bame  results  will  be  obtained  for  more  than  three 
components?  What  limitations  may  b<j  anticipated  when  using  this  technique 
with  more  than  three  components? 
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(2)  Provided  the  actual  response  and  composite  formulation  are  used  in 
equation  (4)  even  though  it  may  be  slightly  different  from  the  lattice  point 
composition,  what  would  be  the  anticipated  effect  on  the  coefficients  What 
limitations  should  be  placed  on  this  difference  to  insure  that  the  response  equation 
would  still  represent  the  true  behavior  of  the  material?  What  effect,  if  any, 
would  this  have  on  the  expected  confidence  limits  of  the  response  predicted  by 
the  regression  equation? 

(3)  Particular  formulations  of  composite  materials  are  difficult  and 
io  me  time  s  almost  Impossible  to  obtain.  For  example,  when  a  small  amount  of 
200  p  filler  particles  Is  used  In  a  composite  formulation  containing  no  40  p  or 

10  p  sizes,  the  200  p  particles  will  tend  to  settle  to  the  bottom  of  the  mold  before 
the  material  la  completely  cured,  resulting  In  a  non-uniform  material.  Provided 
a  given  lattice  point  composition  is  unobtainable,  would  It  be  more  desirable  to 
use  an  estimated  response  for  the  lattice  point  composition  or  to  neglect  this 
point  entirely  in  the  regression  equation? 

(4)  Often  it  becofnes  desirable  to  obtain  an  estimate  for  the  response 
of  composite  formulations  that  are  outside  the  factor  space  represented  by  the 
reference  mixtures  tested.  How  may  the  confidence  limits  be  estimated  for 
those  composition  points  that  are  not  contained  within  the  lattice  of  experimental 
points?  Will  the  procedure  discussed  on  variances  and  confidence  limits  with 
equations  (5) -(13)  still  apply  in  this  case;  If  not,  how  would  one  estimate  the 
confidence  limits  for  these  situations? 

The  range  of  Information  available  from  the  contour  plots,  which  in  this 
study  indicated  the  various  possible  mixtures  ol  filler  particle  sizes  that  will 
result  in  the  same  material  behavior  characteristics,  offers  the  potential  of 
broad  application  in  Army  research  programs,  provided  the  difficulties 
encountered  can  be  resolved.  Simultaneous  consideration  of  a  similar  series 
of  plots  during  the  development  of  any  composite  material  chould  significantly 
improve  the  chances  of  optimizing  the  mechanical  properties  for  a  specific 
design  and  should  greatly  reduce  the  time  and  effort  required. 
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SYSTEMS  VULNERABILITY  DUE  TO  MULTIPLE 
COMPONENT  DRIFT  AND  COMPONENT  FAILURE 


W.  W.  Happ 

U.  S.  A  nay  Construction  Engineering  Research  Laboratory 
Champaign,  Illinois 

ABSTRACT.  Two  distinct  aspects  of  vulnerability  of  a  lumped  system 
are  examined”  (1)  a  smell  performance  drift  par  unit  component  drift, 
commonly  referred  to  es  elasticity  or  sensitivity,  end  (11)  catastrophic 
failure  of  a  component,  commonly  referred  to  as  performance  failure  or 
error.  Sensitivity  analysis  and  analysis  of  failure  are  increasingly 
employ In  computer-oriented  analysis,  optimisation  and  design  of  systems 
and  circuits.  A  systematic  derivation  of  second-ordct  effects  Is  presented, 
as  the  currant  literature  la  restricted  to  first-order  analysis.  Sensi¬ 
tivity  of  sensitivity,  slso  referred  to  as  second-order  sensitivity, 
and  error  of  error,  or  second-order  error,  are  evaluated  as  are  sensi¬ 
tivity  of  error  and  error  of  sensitivity.  Results  are  presented  In  tabular 
form  for  quick  reference.  The  Implementation  of  thesa  concepts  by  data 
manipulation  techniques  such  es  binary  base  tagging  Is  Illustrated.  A 
tutorial  Introduction  to  vulnerability  concepts  is  provi led  to  make  this 
survey  self-contained. 

INTRODUCTION.  This  Investigation  wee  performed  under  the  auspices  of 
the  Coordenajib doe  Programs*  da  Pde-Craduaqfb  de  Engenharla  (COPPE) 
da  Unlvereldade  Federal  do  Rio  da  Janeiro.  The  majority  of  the  material 
is  prasanted  hers  for  th*  first  time  although  some  of  the  materiel  was 
published  as  informal  course  notes  for  a  graduate  seminar  on  "Modelling 
of  System"  at  the  University  of  Wisconsin  in  1966  and  for  a  similar  course 
at  tha  Unlversidad  de  Chile,  1969. 

The  challenge  to  apply  systems  concepts  effectively  to  local  problems 
originated  at  COPPE  end  provided  the  impetus  for  the  development  of 
unified,  fonaal  exposition  of  esnsltivity  and  failure  analysis  as  a  single 
toplct  vulnerability. 

The  systems  analyst  faces  radically  different  kinds  of  problems  In 
countries  with  e  long  tradition  of  successful  technological  development  - 
euch  es  Britain  or  Germany  -  compered  to  problems  faced  by  countries  with 
accelerated  technological  development  -  such  as  Puerto  Rico  or  Venesuala  - 
because  the  latter  expose  themselves  to  Increased  technological  vulnera- 

Acknowlsdgewsnt :  Part  of  chla  work  was  performed  under  sponsorship  of 
USAID.  United  States  Agency  for  International  Development,  as  part  of  a 
tan-week  teaching  and  rsaaarch  assignment  In  Brasil  at  COPPE,  Coordenacao 
doa  Programs  de  Pos-Graduecao  de  Engenharla,  Unlversidade  Federal  do 
Rio  de  Janeiro.  Pert  of  this  statarlal  waa  presented  In  Publlcacao  No. 

10.70  May  1970  of  COPPE  and  was  Issued  as  lecture  notes  In  a  graduate 
course  Slatenaa  multltermlnala.  This  paper  has  also  been  submitted  for 
publication  to  the  Journal  of  System  Science,  England. 
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bllity,  Increased  vulnerability  result*  whan  a  relatively  small  failure 
In  ona  of  cha  alnor  component*  of  tha  system  haa  a  major  affact  on 
overall  ayatan  performance. 

Vulnerability  to  component  failure  or  component  drift  la  usually 
adequately,  and  marly  always  quickly,  recognised  and  compensated  for  if 
an  appropriate  feed  back  component  is  available  to  perform  thla  function. 
Nations  with  a  long  technological  tradition  have  developed  this  feedback 
mechanism.  Countries  without  thia  tradition  have  not  developed  it,  and 
are  therefore  vulnerable,  since  the  pace  of  technological  advance  le 
artificially  accelerated.  Often  this  apsed-up  exceeds  by  orders  of  magni¬ 
tude  the  "natural  mode"  of  technological  development.  It  is  then  that 
the  vulnerability  to  technological  failure  or  dislocation  increases 
rapidly. 

The  resultant  cause-effect  relationship  can  be  formulated  concisely 
in  the  language  of  systems  analysts t  the  system  can  respond  only  in  its 
natural  mode,  tha  delay  for  remedial  action  is  too  long,  and  the  forced 
accelerated  pace  of  technological  development  is  at  variance  with  the 
feedback  mechanism. 

Because  of  the  universality  of  the  laws  governing  system  behavior, 
the  syetema  analyst  is  destined  to  make  a  significant  contribution  in 
resolving  thia  problem.  The  methodology  of  systems  science  requires  only 
minor  adaptations  to  switch  from  the  complex,  lsrge  systems  In  daveloped 
countries  to  entirely  different  systems  as  they  exist  in  developing 
countries.  Beyond  the  intrinsic  academic  incentives  of  analysing  new 
systems,  the  systems  analyst  will  tend  to  study  systems  of  immediate 
practical  value.  This  type  of  analysis  will  thus  be  of  Interest  to  Industry 
and  to  governmental  planning  agencies,  provided  existing  methodology  of 
systems  science  can  be  effectively  extended.  This  le  exactly  the  elm  of 
examining  system  vulnerability. 

Systems  science  applies  analytical  techniques  to  physical,  economic, 
ecological  and  many  other  types  of  syetema,  such  as  communications  systems 
or  project  planning.  The  systems  approach  usually  involves  three  phases: 

(1)  Modelling:  a  description  of  the  system  in  a  form  suitable 
for  analysis 

(11)  Evaluation:  includes  analysis,  optimization,  vulnerability 
and  information  display 

(ill)  Strategy:  search  for  implementation,  definition  of  alter¬ 
natives,  decision  risks,  validation  of  models  and  verifi¬ 
cation  of  process  improvements. 
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The  effective  utilisation  of  systems  science  by  developing  countries, 
hand-in-hand  with  their  acceptance  of  computer  technology,  or  more  epeclf- 
Ically  the  effective  utilisation  of  even  small  computers  requires  increasing 
faalllarlty  with  systems  science  and  tharaby  provides  a  challenge  for 
countries  with  accelerated  technological  development  to  adopt  syataea 
science  to  their  own  needs.  Clearly,  if  computer  technology  is  capable 
of  Increasing  the  paca  of  remedial  action,  the  aim  muat  be  to  develop  a 
strategy  to  identify  and  to  off-set  technological  vulnerability. 

This  investigation  of  vulnerability  has  the  primary  aim  of  making  a 
small  and  modest  atop  In  thin  direction!  to  extend  ays tens  concepts  to 
unexplored  areas  of  vulnerability,  a  topic  of  secondary  Interest  to  research 
agencies  in  highly  developed  countries  with  practically  invulnerable 
technology. 

In  suamary,  computer-oriented  algorithms  for  highly  vulnerable  systems 
are  needed  for  a  simple  reason!  rapidly  developing  economies  are  vulnerable 
systems,  and  no  scholarly  studies  on  this  topic  are  available. 

This  report  is  elso  Intended  to  serve  as  a  guideline  for  a  graduate- 
level  seminar  which  alma  to  apply  vulnerability  analysis  to  electrical 
and  industrial  engineering.  Thus  a  secondary  aim  of  this  investigation  is 
to  stimulate  seminar  projects  in  which  problems  for  graduate  theses  In 
systems  science  and  research  projects  of  practical  Interest  are  formulated. 
Vulnerability  investigations  were  selected  because  they  provide  a  useful 
link  between  (1)  the  urgent  need  by  local  engineers  for  systsms-oriented 
investigations,  and  (11)  the  capability  of  the  tcadmic  coma  unity  to 
provide  computer-oriented  analytical  techniques  to  meet  these  needs. 

The  reader  who  la  not  primarily  interested  In  mathematical  techniques 
may  proceed  directly  to  the  last  section  of  the  report.  The  conclusions 
derived  from  this  Investigation  are  stated  specifically  for  management 
review,  requiring  no  specialised  technical  background.  The  utilisation  of 
the  results  by  economists,  scientists,  engineers,  and  others,  who  are 
primarily  lntarasted  in  possible  applications  rather  than  mathematical 
techniques,  will  thereby  be  facilitated. 


The  remainder  of  this  article  was  reproduced  photographically  from  the 
author's  manuscript. 
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1.  Basic  Concepts  of  Vulnerability 


1.1  Review  of  Current  Literature 

During  the  past  five  years,  techniques  for  sensitivity  and 
failure  analysis  were  developed  for  computer-oriented  analysis 
of  lumped  parameter  networks.  These  techniques  can  be  applied 
to  optimization  in  systems  analysis,  automatic  control,  and 
stochastic  networks.  It  is  therefore  desirable  to  summarize 
existing  results  and  to  extend  them  in  order  to  formulate  a 
systematic  approach.  The  first  part  of  this  investigation  con¬ 
sists  of  a  survey  of  existing  results  based  on  several  recent 
investigations : 

(a)  Development  of  basic  concepts,  analytical  approaches 
with  emphasis  on  scope  and  limitation  of  procedures 
may  be  grouped  according  to  major  techniques: 

•  binary  base  tagging  for  lumped  systems  (1,2,3) 

•  sensitivity  analysis  -  first-order  and  higher 
order  formulae  (4,5,6) 

•  failure  analysis  -  error  and  preassigned  accu¬ 
racy  techniques  (7,8,9) 

(b)  Applications  of  tagging  procedures  to  lumped  parame¬ 
ter  systems: 

•  electrical  circuits  with  active  elements  (10,11, 
12) 

•  stochastic  networks  applied  to  models  of  proce3- 


i  j: 


••i  (13,14,15) 


•  tolerance  analysis  and  Monts  Carlo  techniques 
(16,17,18) 

(e)  Survays  and  stannaries  primarily  with  tutorial  objec- 
tlvsst 

•  tsxtbooks  and  rsfarsnos  t  .Ebooks  (19,20,21) 

•  topical  rsvlsws  and  axposltions  (22-29) 

•  tutorial  survsys  (30-35) 

1.2  Ttralnoioqy  for  Lumped  Sy  a  tarns 

Consldar  tha  circuit  in  Fig.  1,  in  which  Z^,  Z2  and  Z3 
ara  known  impedance,  from  which  tha  unknown  impadanca 


*x*i  ♦  Vj 

“l  *  r2  +  * 


3 


is  to  ba  found.  Tha  arbitrary  minus  sign  for  tha  unknown 
will  permit  us  to  writs  tha  result  as  Z  ■  0  whara 


8  "  *^3  ♦  S^Z4  ♦  ZjZj  ♦  ZjZ4  or  Z 


*ila‘«*1a(»iJ(»i4 


where  d(l)  is  saro  or  ona  and  obeys  tha  ooda  pattern  V  •  I  or 
?  »  8.  Symbolically, 


10  10 

and  Y*  ■ 

0  10  1 

10  0  1 

0  110 

0  110 

10  0  1 

0  10  1 

10  10 

0  0  11 

110  0 

(4) 
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Not*  that* 


<•)  Inspection  of  FI9.  1  show*  that  th*  cod*  Z  can  be 
obtained  directly  as  the  cut  aat  from  th*  geometry. 
Similarly  coda  Y  above  represents  the  tie  set. 

(b)  The  unknown  Z4  plays  the  same  role  as  the  known 
quantities.  Unknowns  are  isolated  by  expansion  of 
Z  -  0;  namely, 

Z  -  Z(24  -  0)  «■  Z4Z(Z4  ■  1) 
whera  Z(Z4  ■  0)  ■  *1*3  +  Z2Z3  scanning  Z*  for  Z4 
and  Z(Z4  >1)  ■  Z1  +  Zj  ♦  also  scanning  Z*  for  Z4 
hence  Z4  -  -Z{Z4  -  0)/2(Z4  -  1) 

(c)  Similarly  Y  -  0  yields  Y4  -  -Y(Y4  -  0)/Y(Y4  -  1) 

This  is  conveniently  written  as  Y4  •  -Y(?)/Y(4*) 

1.3  Illustrative  Example  of  Vulnerability 
To  find  Z4  when  Zj  •  0  obtain  from  Z  ■  0  and  Z* 

Z(Z.  -  0,  Z,  -  0)  Z.Z, 

Z .  (Z«  -  0)  - - —3 - £ -  -  -  1  -3-  - 

4  2  Z(Z4  -  1,  Z2  »  0)  Zx  +  Z3 

The  system  performance  when  component  2  falls  1st 

Z(4,I)  "  r where  Z (T, 2) ■  Z.Z, 

Z  (4  ,i)  1  3 

and  Z{4',?)«  Zx  ♦  Z3 
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Generally  If  P  and  Q  balong  to  a  aystain  H  •  0  than 
H(P?Q)  ■  0  ylald  for  P  with  Q  sat  to  saro 

P(5)  -  - 

H(P',0> 

Similarly  P  with  1/Q  »at  to  saro  yialda 
P(Q')  •  -  H ) 

'  h<p\5> 

Thus  with  1/Z2  sat  aqual  to  saro  yialds 

S.(S2  -  1)  -  -  liLilL-  .  z 
4  2  Z (4 '  / 2' )  3 

whara  Z(T,2')  -  Zj  and  Z(4',2')  ■  1,  by  scanning  Z*, 

Sines  H (P,Q)  -  0  yialds 

P  -  -  H (P)/H (P ' )  and  Q  -  -  H(Q) /H(Q* ) 


and  da  -  0  yialds  H<Q')dQ  ♦  H(P*)dP  »  0 

d  InP  „  dP  _  H{Q* )  H(5)  H(P') 
d  InQ  dQ  P  H (P ' )  H(Q')  H(P) 


hanea  S(P,Q)  -  -H(5)/H(P) 
Por  axanpla  S(Z4,Z2)  »  -  « 


Z^Z Z^Z^ 
Z2Z3*  ZjZ3 


sines  Z(T)  ■  Z2Z3+ 


Zft)  -  Z^Zj*  Z3Z4-»-  ZXZ4 
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are  first-order  vulnerability  effects  i 
(a)  sensitivity  or  drift  of  z4  with  Z2  la  dafinad  by  the 
fractional  or  percentage  ohanga  in  Z4,  if  Z2  ia 
changad  by  ona  par cant,  namaly 
(dz4/z4)/(dz2/z2)  -  Z(I)/Z(T) 

Of tan  alao  rafarrad  to  aa  elasticity,  the  first- 

ordar  affact  ia 

d  InP  .  _  H(5) 
d  InQ  H  (F) 


(b)  failure  of  component  Z2  af facta  the  performance  para¬ 
meter  Z4  of  the  ayatara  either  by  "open"  or  "abort"  of 
the  element.  Thus  error a  due  to  two  kinds  of  catas¬ 
trophic  failure  can  be  readily  confuted  from  Z* , 
namely 


2(4,3) 


SJLIL 

Z(4',I) 


and  Z (4 ,2 1 ) 


Z(T  .2') 

Z (4’ ,2') * 


A  sharper  definition  of  these  concepts  will  now  be 
attempted,  followed  by  an  extension  to  second-order 
oonoepts . 
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2*  Systematica  of  Flrat-Ordar  Relationships 


a 

7 


2.1  Sensitivity 

Consider  a  linear  ayatem  in  which  two  alamanta  P  and  Q  sa¬ 
tisfy  tha  aquation  H (P,Q)  *  0  whara 

H  -  H<p,5)  ♦  PH(P',5)  ♦  QH(F,Q’)  «■  PQH(P’  ,Q')  -  0  (1) 


The  aanaitivlty  of  P  with  raapect  to  Q  ia  dafinad  aa 


S(P,Q) 


d  In  P 
d  In  Q 


(2) 


Since  H($)  ■  H(F,5)  ♦  PH(P',5)  it  ia  often  desirable  to 
eliminate  P  from  the  expression  for  sensitivity: 


S(P,Q) 


H  (0)  ^  H (P '  ,5)  »H(g)“H(P,5)  *H (P * ) 
H(F)  H(P)*H(P‘) 

.  Htf'.g)  .  H(P,5) 

B(P')  H(F) 


In  circuit  analysis,  it  is  often  necessary  to  find  the 
sensitivity  with  respect  to  1/P  with  respect  to  Q.  It  ie  con¬ 
venient  to  denote 

S(P\5>  -  H(P’,5)/H(P'>  and 

8 (F,8)  •  H(F,5)/H(F)  hanoe 

8<J,Q)  -  8<P\5>  -  8<F,5)  and  (3) 

S($,Q)  -  -  S(P,Q)  -  S(F,Q)  -  S (P' , Q)  (4) 
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In  aquations  (3)  and  f  *'  ♦•he  tarns  on  tha  right  hand  slda 
do  not  depend  on  tha  vali  ~i  »  and  tharafora  maka  aval labia 
by  tagging  only  known  symbc-j .  To  darlva  axprassions  of  this 
typa  for  sansltlvlty  and  arror  functions  is  tha  purposa  of 
this  invastlgatlon.  Note  tha  similarity  In  tha  function  S  (5,5) 
and  E  (P,C) . 

2.2  Error  Qua  to  Single  Failure 

Dafina  tha  arror  in  P,  with  Q  -  0  (a  short  circuit  if  Q  is 
an  impedance)  by 

E(P*,Q)  ■  (5) 


and  tha  arror  in  P,  with  Q  - 
impedance)  by 

E(P',Q')  -  P. 

P 

It  follows  that 

E  (P '  ,Q)  - — 3® 

H(Q)  -  H(P,5) 


(an  open  circuit  if  Q  is  an 


(6) 


(7a) 


and 


E(P',Q')  - 


JUflll 


H(Q’)  -  H(P,Q') 

Similarly,  define  the  errors  in  p  by 

1 _ 1_ 

k(F#C)  -  g  ~  t(OI 

f 


(7b) 


(8a) 
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i-rwii"  r“>9awt'  H''*** 1  !*W' 


K  tip..  |  g/g  1  |Mli||r||  Bga 


B(f,Q')  - 


leading  too 

E(P,Q)-  -5^  (9m 

h(P,5)  (,a 

tif.Q')  •  ■#%{  (9b 

H (P,Q  ) 

These  expressions  ere  not  yet  useful  for  tagging  since  H(5) 
and  H(Q')  depend  on  the  value  of  P,  but  substituting 


H(5)  *  H(P,5)  -  *  H(P’,5) 


(10a) 


H(Q')  -  H(P,Q')  -  •  H(P'  ,Q' ) 

H  (P 1 ) 


(10b) 


yields  the  following  useful  result: 


B(P',X)  -  1  - 


H(P.X)  ■  H  (P 
K  (P '  ,X)  Hfl?) 


(11a) 


s<*.x)  -  i  -  x)  ■ 

H(F,X)  •  H(P') 

where  X  can  be  either  5  or  Q* . 


(lib) 


2.3  Error  Due  to  Multiple  Failure 


It  is  useful  to  determine  the  error  in  P  or  p  if  two  ele¬ 
ments,  Q  and  R,  were  to  fall* 
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Zb  general, 


B(P*>X,Y)  ± 


P  -  KX,Y) 


(12*) 


B(*|X,Y)  a 


(12b) 


etiere  X  ■  5  or  Q'  and  Y  -  K  or  R'. 

The  final  result  is  analogous  to  the  single  failure  case: 


KP-.x.m  -  i  -  "F'X'W-  ■  mil 

mt'M  Hff> 

I(p,x,«  - 1  -  .  ajB- 

H  (P,X,  Y)  H(P') 

2.4  Sensltlvlty-Error  Relations 

Consider  E(P,5)  -  **(5). 

H(P,0) 

B(F,Q*)  - 

H(F,Q’) 


(13a) 


(13b) 


Then 


1 _ 1 

B(F,0*)  E(F,5) 


SiLall  .  HiML  .  s  (Q#P) 
H(O')  H<5)  tQ'r' 


so  that 


1 _ 1  .  l 

B(F,Q‘)  B(F,5)  S(P,Q) 
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3,  Second-Order  Relationships 


3,1  Higher  Order  Sensitivity 

Define  the  '‘sensitivity  of  a  sensitivity”  by 

S(S(P,Q)  ,  R)  =  — 5^2 

d  InR 

It  has  been  shown  (9)  that 


S(S(P,Q)  , R) 


H  (P.R) 
H  (P) 


H  (Q/R) 
H  (Q) 


yielding,  after  simplifying,  the  result  given  in  table  2, 
3.2  Higher  Order  Error 
Define  the  error  of  an  error  by 


E(E(P',Q)  ,R) 


E(P',Q)  -  E^?’,Q,R) 
E(P',Q) 


Recall  that  E(P',Q)  is  the  error  in  P  when  Q  =>  0,  and 
E(P'jQ,R)  is  the  error  in  P  when  Q  =  R  =  0. 

In  general, 

e(e(p',x),y)  a  ?xfy) 

£>  \P  , X) 


and 

E(E(P,X),Y)  A  2-E(P;XfY) 

E  (P ,  X) 
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(15) 


(16) 


(17) 


(18a) 


(18b) 


Substituting  (7)  and  (9)  In  these  axprassions  and  than  ax< 


pending  all  terms  which  deoend  on  P  by  tha  formula 


H(X)  -  H(?,X)  -  tJiS-  H(P\X) 


yialda  axprassions  usaful  for  computer  analysis,  given  in 
tabla  2. 

3.3  Sensitivity  of  Error 

Define  tha  sensitivity  of  an  error  by 


S(E(P*  ,Q)  ,R)  A 


d  In  E  (P ' 


Thus, 


S(E(P',Q),R)  -  — ^5^*  •  -7 

d  R  E  (P  #0) 


Using 


B(p., 5,  H(5) _  d  d  H(X) 

1  ,0’  H(5)  -  h(p,5)  ana 


H  (X  #  R) 


dE(P’.g) 
d  R 


[§)-H  (P,Q)  }-H{Q) 
{H{Q)-H(P,Q)  }2 


{H  (Q)  -  H(P,g)}2 


_ 5 _  .  (?  /  Q  y  R).-H  (P  ,Q)  H  (Q  ,  R)  )  :  R 

E  (P '  »Q)  <H(Q)-H(P,Q)  }{H(Q)  ) 


J 


To  got  a  result  useful  for  tagging,  simply  expand  H (Q) 
and  H(Q,R)  by  (20)  and  use 


and  then  expand  H(5)  and  H(R')  by  (19). 

The  final  result  Is  given  in  table  3,  with  similar  results 
for  S(E(Pf&),R)  as  well  as  the  analogous  case  for  Q1  in  place 
of  5. 


3.4  Error  of  Sensitivity 

Define  the  error  of  a  sensitivity  by 

E(S(P,Q),R)  t  Sil>./-^~~^<PfQrR) 


(21) 


where 

S(P,Q,R)  a  S(P,Q)  I 

I  R-0 

Through  computations  analogous  to  those  used  in  deriving 
the  expressions  for  error  of  error,  the  final  results  in  table 
1  are  obtained. 

4.  Discussion  of  Binary  Base  Tagging 

Table  5  lists  terms  obtained  from  an  expansion  of  H  ■  0  in 
terms  of  three  parameters  P,  R  and  Q.  If  the  expression 
H(P,Q,R)  is  tagged  in  terms  of  these  parameters,  the  second" 

order  effects  evaluated  in  Tables  1-4  are  obtained . 
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5.  Perspective :  System  Vulnerability  due  to  Unreliable 

Components 

The  vulnerability  of  a  system  consisting  of  numerous 
interacting  components  is  measured  by  the  change  of  a  speci¬ 
fied  performance  parameter  of  the  system  (P)  caused  by  a  unit 
change  in  a  component  value  (Q) .  The  classical  example ,  ex¬ 
amined  by  Bode  (1)  in  1941  is  the  change  in  current  amplifi¬ 
cation  (P)  of  an  electronic  circuit,  if  the  transconductance 
of  the  vacuum  tube  (Q)  is  changed.  Although  Bode  does  not 
explicitly  define  two  distinct  types  of  vulnerability,  two  in¬ 
terrelated  aspects  of  vulnerability  are  recurring  frequently 
in  his  work: 

Sensitivity  or  drift  S(P;Q)  is  defined  as  the  frac¬ 
tional  or  logarithmic  change  d(ln  P)/d(ln  Q)  of  the 
performance  parameter  (P)  of  .the  system  caused  by 
corresponding  change  of  the  component  parameter  (Q)  . 
Mathematics  texts  refer  to  S(P;Q)  as  "elasticity", 
while  the  electronics  terminology  refers  to  small 
changes  of  component  values  affecting  system  perfor¬ 
mance  as  "drift". 

Failure  of  a  component  (Q)  affects  the  system  perfor¬ 
mance  differently  and  is  measured  by  the  error  in  sys¬ 
tems  performance  E(P;Q).  The  error  E(P;Q)  is  caused 
by  catastrophic  failure  of  Q  and  affects  the  system 

performance  parameter  (P) .  Failure  of  a  component  in 
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an  energy  transforming  system  esually  results  in  one 
Of  two  predictable  extremum  conditions  of  the  compo¬ 
nent.  In  circuits  these  conditions  consist  of  open- 
circuit  or  short  circuit  failure  of  a  component.  In 
probabilistic  networks  the  extrema  are  certainty  of 
occurrence  or  of  non-occurrence  of  an  event. 

Although  a  large  number  of  investigators  have  drawn 
heavily  on  Bode's  work,  the  overwhelming  majority  of  investi¬ 
gations  are  devoted  exclusively  to  one  type  of  vulnerability, 
primarily  sensitivity*  Despite  an  extensive  literature  on  sen 
sitivity,  r..ost  investigation  fails  to  correlate  advances  in 
sensitivity  with  corresponding  advances  in  failure  or  “error" 
analysis . 

About  twenty  years  elapsed  between  node’s  fundamental 
work  and  the  next  major  step  (2,3):  the  exploitation  of  the 
isomorphism  between  graph  theory  -  referred  to  by  engineers  as 
"topological  techniques"  -  and  vulnerability  -  frequently  ex¬ 
amined  by  engineers  as  part  of  "reliability  analysis". 

During  the  past  decade  (1960-1970),  automated  evalua¬ 
tion  techniques  have  extended  vulnerability  calculations  sig¬ 
nificantly.  The  size  of  a  system,  which  can  be  analyzed  by 
an  economically  purposeful  effort  has  increased  by  two  orders 
of  magnitude.  For  systems  with  lumped  components,  for  exam¬ 
ple  an  electronic  circuit,  conventional  techniques  are  concerned 
with  systems  of  5  to  10  components,  while  computer-oriented  tech 


niques  are  attempting  corresponding  calculations  for  systems 
with  20  to  50  components.  This  change  in  design  philosophy 
has  triggered  a  vigorous  search  to  extend  vulnerability  con¬ 
cepts.  Three  examples  will  illustrate  this: 

Preassigned  accuracy  techniques:  A  computer-aided  or 
computer-oriented  selection  cbtains  from  the  original 
system  a  simplified  or  an  approximate  system  by  elimi¬ 
nating  components.  The  approximate  system  must  be 
capable  of  supplying  performance  data  with  a  pro -spe¬ 
cif  ^nd  accuracy.  To  illustrate,  assume  that  a  system 
with  200  components  is  described  by  a  performance  pa¬ 
rameter,  which  is  needed  with  only  10%  accuracy.  Ex- 

t  nan 

cept  for  pathological  cases,  -s  v 1C  components  can 
each  contribu  .e  more  than  10%,  to  the  systems  perfor¬ 
mance.  Thus  it  a  likely  that  20  components  will  pro¬ 
vide  a  model  to  describe  the  system  within  the  preas¬ 
signed  accuracy.  An  aaequate,  more  economical  model 
is  thus  established. 

Binary  base  tagging:  A  scanning  procedure  for 

1 dent i f  vins 

efficiently  groups  of  binary  symbols,  is  used  to 
replace  far  lengthier  numerical  calculations.  An  ap¬ 
plication  was  presented  ht-r-.-  usir.v  this  technique  for 

vulnerability  calculation.  Since  vulnerability  calcu¬ 
lations  by  their  very  nature  involve  second  order  ef¬ 
fects  in  which  small  differences  are  of  paramount,  im- 


portance#  numerical  calculation!  are  subject  to  signi¬ 
ficant  round-off  errors.  Xf  these  errors  are  cumula¬ 
tive#  iterations  may  lead  to  numerical  instability. 
Symbol  manipulations  lead  to  calculations  which  in 
practioe  exclude  the  possibility  of  numerical  instabi¬ 
lity  and  its  inherent  difficulty  of  detection.  Tag¬ 
ging  and  similar  symbol  manipulation  techniques  are 
preferable 

(i)  when  the  numerical  stability  of  calcula¬ 
tions  is  in  doubt 

(ii)  when  parametric  solutions  are  required, 
thus  maintaining  some  symbols  and  repla¬ 
cing  others  by  numbers 

(ill)  when  special  techniques#  such  as  vulnera¬ 
bility  methods#  can  be  formulated  more 
efficiently  by  symbol  manipulation  than 
by  numerical  approaches#  for  example  dif¬ 
ferentiation  of  a  function. 

Worst  case  design t  The  effect  of  all  possible  simulta¬ 
neous  parameter  variations  are  combined  to  ascertain 
which  combination  yields  the  widest  fluctuations  for  a 
specified  performance  parameter.  It  is  difficult  to 
anticipate  the  effect  of  the  variations  of  one  parame¬ 
ter#  on  a  specified  system  performance  parameter#  since 
several  components  are  statistically  likely  to  cause 
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changes  which  compensate  each  other .  Conventional 
optimisation  tachniquas  employ  successive  approxima¬ 
tions  to  arrive  at  a  unique  selection  of  parameters. 
Alternatively  it  is  possible  to  develop  appropriate 
properties  of  vulnerability*  which  will  assure  in  each 
instant  that  the  "worst  case"  for  each  component  is 
selected  for  obtaining  the  specified  effect  on  the 
system. 

These  three  examples  merely  illustrate  the  range  of 
applications  of  vulnerability  analysis*  which  emerges  as  an 
expanding  but  nearly  unexplored  frontier  in  systems  science. 
The  definitions  and  concepts  of  second  order  effects  were  ex¬ 
amined  and  basic  formalisms  were  developed.  Higher  order  ef¬ 
fects  are  still  one  of  the  unexplored  areas  deserving  atten¬ 
tion. 

This  investigation  aims  st  being  the  first  in  a  se¬ 
quence  of  reports  on  vulnerability  studies.  Subsequent  inves¬ 
tigations  will  apply  these  formalisms  developed  here  to  multi- 
tenninal  networks*  thermodynamic  fluctuations  and  similar  phys 
leal  systems  as  well  as  to  modelling  of  production  flow  and 
to  procsss  control. 
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Basic 

TABLE  1 

m 

Niparator  00 

Denominator  (D) 

l-EOP'-.Q) 

H(P,Q)  •  H(P’) 

R(P\Q)  *  H(P) 

l-E(P’iQ') 

H(P,Q’)  •  H(P’) 

• 

H(P* ,Q*)  ‘  H(P) 

1-E(P;Q) 

H(P\Q)  •  H(P) 

H(P,Q)  *  H(P') 

1-E(P;Q' ) 

H(Pf ,Q')  *  H(P) 

H(P,q*)  •  H(P’) 

Jh(p'.q)  H<P,Q)I 

H<P)  •  HG*> 

S(P;Q) 

Ih(p')  h<p)  1 
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TABLE  2 


Sacocd-Ordar  Error  and  Sanaitivity 


N/D 

1~E(E(P ' ;Q) ;R) 

H(P\Q) 

1-E(E(P{Q);R) 

H(P,Q) 

1-E(E(P';X);Y) 

H(P',X) 

1-E(E<P;X);Y) 

H(P,X) 

S(S(P;Q) ;R) 

H(P,R)  - 

R(P)  * 

Nuaarator  (N) 


|H(P,Q,R) 

•  h(p',q,rJ 

Ih(p) 

H(P’)  1 

IH(P,Q,R) 

h(p’,q,rI 

Ih(p) 

H(P’)  1 

|H(P,X,Y) 

H(P\X,Y)I 

|H(P) 

H(P')  1 

|H(P,X,Y) 

H(P\X,Y)1 

|hcp) 

H(P’)  1 

H(P,Q) 

H(P\Q)  J 

H(P) 

H(P')  f 

|K(P,Q,R) 

HCP’.Q.R)! 

IH(P) 

H(P’)  j 

panoainator 

121 

...  _  |H(P,Q) 

H(p’,d 

H(P’,Q,R)  -I  _ 

|H(P) 

H(P')  I 

_  fH(PeQ) 

H(P\qJ 

H(P»Q»R)  *1  _ 

i 

|H(P) 

H(P')  \ 

jH(P,X) 

H(P\X)| 

H(P'.X.Y)  *1 

j 

|H(P) 

H(P')  1 

|H(P,X) 

H(P\X) 

H(P,X,Y)  .1  _ 

«H(P) 

H(P') 

f H(P,Q) 

H(P\Q> 

H(P)  .  1 

Jh(p) 

H(P’) 
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M/D 

1-E<S(P;Q);R) 

1-E(S (P ;Q) ; R* ) 


TABLE  4 

Error  in  Sensitivity 
Numerator  (N) 

IH(P,R)  H(P',R) 

_  _ 

H(P.Q,R)  H(P',Q,R) 


H(P) -H(P' ) 


H(P,R' )  H(P',R') 

;H(?;Q,R')  H(P  '  ,Q,R'  )j 


H(PlR)-H(P’,R) 


Denominptor  (D) 

H(P)  H(P') 
H(P.Q)  H(P\Q) 


H(P’) 


H(F,K!)-H(P’,R')*  _ 

H(P,Q)  H(P ' ,0) 


E(P';Q) 

EfP'jjQ') 

E(P;Q) 

E(P';a'l 

E(P’;2.R) 

E(P';2,R') 

E(P';Q,P.') 

EfP'iQiH') 

E(Pifi.K) 

E(P;Q,RD 

ECPiQ’.R) 

E(P;Q’R’) 

S(PiQ) 

B(E(P’  ;£);R) 

E(E(P' ; Q) ;R' ) 

E(E(P';Q');R) 

E(E(P'  .q'X;R' ) 

E(E(P;S);R) 

E(E(P;Q);Ri) 

E(E(P,Q');R) 

E(E(P;Q')R') 

S(S(P;q2.;R) 

S(E(P';Q);R) 

S(E(P'j.Q');R) 

S(E(P;Q);R) 

S(E(P;q');R) 

E(S(P;Q) ;R) 

E(S (P ; Q) ; R' ) 
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TIME  CONSTRAINED  RELIABILITY  DATA  DEVELOPMENT  FOR 
RADIO  EQUIPMENT  IN  A  GROUND-BASED  LABORATORY 


C.E.  Deekard  and  T. K.  DeClue 
Wyle  Laboratories 
Huntsville,  Alabama 


ABSTRACT 


A  ground-based  laboratory  test  program  was  conceived  for  the  purpose  of  economically 
acquiring  a  qualitative  measure  of  the  MTBF  of  radio  equipment  subjected  to  a  helicopter 
combat  flight  environment.  The  initial  portion  of  the  program  was  time  constrained  with 
a  very  small  number  (two)  of  test  specimens  available.  This  dictated  departure  from  the 
normal  single-axis  testing  technique.  A  system  designed  to  provide  testing  in  the  three 
orthogonal  axes,  simultaneously,  was  implemented  end  utilized  to  acquire  the  desired  data. 


INTRODUCTION 

A  requirement  for  determining  the  system  effectiveness  of 
the  AHl/ARC  115  Radio  Set  (used  in  the  OH-58  Helicop¬ 
ter)  was  recently  set  forth  by  the  United  States  Aviation 
Test  Board,  Fort  Rucker,  Alabama.  At  present,  flight 
tests  are  being  conducted  to  obtain  the  data  required  for 
establishing  the  equipment  MTBF,  The  execution  of 
these  flight  tests  demands: 

e  Extensive  use  of  personnel  and  aircraft. 

e  Schedule  difficulties  due  to  aircraft  and 
flight  crew  priorities. 

e  High  costs. 

It  is  felt  that  system  effectiveness  may  be  economically 
determined  by  subjecting  the  radio  set  to  a  controlled 
series  of  dynamics  tests  in  a  ground-based  laboratory. 

The  advantages  of  laboratory  testing  are 

e  Control  of  conditions 
e  Control  of  test  scheduling 
e  Simplification  of  analysis 
e  Repeatability 
e  Economy 
e  Safety 

A  high  degree  of  simulation  realism  would  be  imperative 
in  such  an  approach;  the  primary  system  requirement 
would  be  the  development  of  a  test  system  capable  of  im¬ 
posing  the  actual  flight  dynamics  inputs  in  the  specimen’s 
three  orthogonal  axes  simultaneously.  Such  a  system 
should  provide  the  data  necessary  for  developing  on 
accurate  measure  of  the  equipment  MTBF  since  it  would 
effectively  reproduce  the  service  environment.  The 
program  under  discussion  can  be  called  a  "pilot  program, " 
the  primary  purpose  of  which  is  to  determine  both  the 


feasibility  and  cost  savings  to  be  realized  by  ground- 
based  laboratory  testing - 

The  purpose  of  this  paper  is  to  describe  the  test  system 
designed  to  provide  the  dynamic  inputs  that  would 
effectively  simulate  actual  combat  flight  conditions. 

This  system  is  presently  operational.  Saving  provided 
approximately  775  hours  of  flight  simulation  to  the  radio 
equipment.  Though  more  data  are  required  to  make  a 
final,  detoiled,  comparison  between  the  flight  and 
ground-based  tests,  excellent  results  have  been  obtained 
in  the  laboratory.  At  present,  the  number  of  radio  sets 
under  test  has  been  increased  from  two  to  six. 

Though  the  primary  intent  of  the  paper  is  to  describe  the 
test  system,  the  general  design  philosophy  used  in  de¬ 
veloping  the  system  is  also  discussed.  Certain  portions 
of  the  design  presented  herein  are  proprietory  with 
patent  rights  pending;  although  the  techniques  inherent 
in  the  design  philosophy  are  available  to  those  who  wish 
to  use  Them. 

TEST  SYSTEM  DESIGN  PHILOSOPHY 

The  procedures  employed  in  the  design  of  a  dynamics 
test  system  begins  with  an  evaluation  of  the  important 
aspects  of  the  service  environment,  viz.  the  origins, 
transmission  paths  and  coupling  modes  of  the  forces  act¬ 
ing  on  the  specimen  during  service.  The  primary  objec¬ 
tive  is  to  assess  the  mognitudes  ond  frequency  spectra  of 
these  forces  ond  couple  them  to  the  specimen  with  im¬ 
pedances  similar  to  those  of  the  mountings  ond  supports 
used  in  actual  service.  When  possible,  the  actual  sup¬ 
port  hardware  should  be  used  in  the  testing  configuration 
in  order  to  provide  o  more  exact  duplication  of  the  ser¬ 
vice  boundary  conditions.  The  more  complex  the  test 
item  and  its  support  hordware  ore,  the  greater  is  the 
need  for  accurate  boundary  reproduction  and  the  more 
difficult  it  is  to  provide  simulation  of  the  boundary 
through  manipulation  of  the  inputs  and  test  fixtures. 
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An  initial  step  in  the  design  of  the  test  system  is  to 
determine  How  large  o  section  of  the  entire  system,  of 
which  the  specimen  is  a  part,  con  be  accommodated 
within  the  laboratory  space  ana  exciter  force  limits. 
Secondly,  determination  of  the  number  of  degrees  of 
freedom  of  motion  that  the  specimen  experiences  most  be 
made.  V/ here,  through  analyses  or  data  examination  of 
the  actual  environment,  it  Is  found  that  motion  is  insigni¬ 
ficant  or  does  not  occur  in  or  about  one  or  more  of  the 
axes,  those  degrees  of  freedom  may  be  restrained  without 
adversely  effecting  the  test  system  or  -esulting  test  data. 
The  design  and  construction  of  a  six  degrees  of  freedom 
system,  although  conceivable  with  present  day  technology 
fends  to  be  extremely  expensive.  Since  the  rotational 
axes  in  typical  flight  environments  generally  experience 
minimal  frequency  and  amplitude  response,  consideration 
should  be  given  to  the  three  translational  axes.  Thus, 
the  design  of  a  test  system  capable  of  excitation  in  the 
three  translational  axes  (while  restraining  the  three 
rotational  axes)  would -allow  the  use  of  available  single 
degree  of  freedom  elecfrodynamic  exciters  at  modest 
expense . 


and  the  event  time.  A  lk  Hz  acoustic  calibrator  was 
used  to  produce  a  constant  amplitude  audible  tone  for 
periodic  transmitter  modulation  checks  during  the  flight 
tests. 

A  total  of  two  complete  flight  profiles  were  performed 
during  flight  testing.  These  profiles  were: 

•  Flight  Profile  No.  1 

-  One  minute  hover  (3  feet) 

-  Seven  minute  flight  ot  maximum  velocity 
attainable,  not  ta  exceed  VNE  (velocity 
never  exceed)  at  1500  ft  M5L  (mean  sea  level) 

-  Simulated  diving  firing  with  60  degree  bank, 
180  degree  side  entry  and  right  pull  up 

-  Twenty  minute  loitering  ot  60-70  knots 

-  Diving  firing  with  60  degree  bonk,  180 
degree  side  entry  and  right  pull  up 

-  Twenty  minute  loitering  at  60-70  knots 


DETERMINING  THE  ENVIRONMENT 


-  Simulated  diving  firing  with  60  degree  hank, 
180  degree  side  entry  and  right  pull  up 


An  OH-58  Helicopter  was  instrumented  on  and  about  the 
AN(/ARC  1 15  radio  set  with  a  total  of  seven  piezoelectric 
accelerometers  for  the  purpose  of  monitoring  fhi  in-flight 
vibratory  environment.  Figure  1  illustrates  the  helicopter 
flight  control  panel  and  the  accelerometer  locations. 


Figure  1,  Helicopter  Flight  Control  Panel; 

AN(/ARC  115  Radio  and  Monitoring 
Accelerometers 


A  14-channel  tope  recorder  was  installed  onboard  the  air¬ 
craft  to  record  all  accelerometer  output  signals.  During 
the  flight  tests,  one  channel  was  reserved  for  voice 
commentary  to  cover  both  the  general  test  conditions 


-  Twenty  minute  loitering  or  60-70  knots 

-  Diving  firing  with  60  degree  bonk,  180 
degree  side  entry  ond  right  pull  up 

-  Twenty  minute  loitering  ot  60-70  kno*s 

-  Simulated  diving  firing  with  60  degree 
bonk,  180  degree  side  entry  and  right  pull 
up 

-  Twenty  minute  loitering  ot  60-70  knots 

-  Diving  firing  with  60  degree  bonk,  180 
degree  side  entry  and  right  pull  out 

-  Seven  minute  flight  at  maximum  velocity 
attainable  not  to  exceed  VNE  at  1500  ft 
MSI 

-  One  minute  hover  (3  feet) 

Flight  Profile  No.  2 

This  mission  wos  conducted  in  the  some  manner 

OS  Flight  Profile  No.  1  with  the  following 

exceptions: 

a.  The  simulated  diving  firing  ond  divirxj 
firing  order  were  reversed. 

b.  All  diving  pull  ups  were  mode  to  the  left. 


Vibration  data  were  continuously  recorded  during  take-off* 
dash  to  VNE,  each  phose  of  the  flight  profiles  ond  land¬ 
ings,  transmitter  carrier  output  and  modulation  were 
periodically  measured  and  recorded  throughout  each  flight 
test.  At  the  conclusion  of  each  flight  test,  the  toped 
data  were  played  back  into  a  recording  oscillograph  for 
visual  display.  This  allowed  examination  of  the  recording 
process  and  provided  a  means  of  determining  the  need  for 
possible  retest. 
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The  vibration  data  obtained  during  the  actual  flight  test 
were  reduced  to  X-Y  plots,  oscillograph  records,  and 
computer  analyzed  power  spectral  density  (PSD)  plots. 
These  data  were  analyzed  and  reviewed  with  cognizant 
Fort  Rucker  personnel.  Figure  2  shows  typical  PSD  plots 
of  accelerometer  data. 

Those  channels  that  displayed  the  highest  levels  of  vibra¬ 
tion  in  the  three  orthogonal  axes  of  the  radio  set  were 
isolated  and  selected  for  providing  the  dynamic  npuls  into 


the  test  system.  A  master  tape  lecording  was  piepared  from 
these  selected  channels.  The  master  tope  contained  the 
most  dynamically  severe  portions  of  the  two  flight  profiles 
flown.  Thus,  o  basic  test  system  input  profile,  approxi¬ 
mately  two  hours  in  length,  wos  developed.  As  anticipa¬ 
ted,  the  most  severe  dynamic  environment  occurred  when 
the  helicopter  was  diving  and  firing  its  weapons  system 
simultaneously.  This  (light  phase  occupied  approximately 
0.5  pet  cent  of  the  two  hour  test  profile. 
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Figure  2.  Typical  PSD  Plots  of  Accelerometer  Data 
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EVALUATION  Of  THE  ENVIRONMENT 

Having  defined  the  dynamic  environment,  it  now  became 
necessary  to  evaluate  the  environment  and  those  techni¬ 
ques  that  would  most  realistically  reproduce  the  environ¬ 
ment  in  the  laboratory.  It  was  realized  that  an  ordinary 
set  of  three,  uni-axial  vibration  tests  would  not  provide 
the  data  that  is  required  for  estimating  the  service  life 
or  MTBf  of  an  item,  in  as  much  os  cross-axes  coupling 
con  couse  failures  not  reproducible  in  single  degree  of 
freedom  testing.  In  reality  a  set  of  tests  of  N  hours 
duration  in  the  three  orthogonal  axes  may  result  in 
either  a  greater  or  less  severe  environment  for  sub -com¬ 
ponents  located  within  the  test  item  than  would  N  hours 
of  rhe  actual  service  environment .  Also,  where  it  is 
possible  that  'rattle  space'  problems  might  occur,  uni¬ 
axial  testing  does  not  exercise  oil  the  internal  degrees 
of  freedom  of  rhe  device  simultaneously,  as  does  multi  — 
axial  testing. 


If  the  component  is  mounted  such  that  its  body  axes  are 

parallel  to  the  three  orthogonal  vibration  axes  ordinarily 

selected  for  sets  of  single-axis  tests,  and  the  single  axis 

vector  inputs  are  applied  to  the  three  axes  for  an  equal 

length  of  time,  N,  the  fatigue  is  proporationol  to 

I  =  00  KNZ)  -•  (3.16  KNX)  *  (  KNY). 
o 

If  however,  the  component  body  oxes  wete  all  rotated, 
say  45°,  from  the  input  axes,  the  fatigue  would  be  pro¬ 
portional  to: 

I  =  300  KNZ  sin  o  ■*  3.16  KNX  sin  .3  -  1  KNY  sin  7) 

The  I  for  the  first  cose  is  14.16  KN  whereas,  for  the 
c 

second  cose,  whore  sin  a  *  sin  3  =  sin  7  '  .707,  the 
sum  it  29.39  KN.  The  difference  of  2:1  !i,  of  course, 
explained  by  the  resolution  of  the  input  vectors  into  body 
reoction  vectors  and  responses. 


Consider  a  typical  electronic  component  mounted  to  a 
surface  (Figure  3)  by  bending  its  axial  leods  at  right 
angles  to  the  body  and  securing  the  free  pads. 


tf  the  three  Inputs  in  the  three  axes  were  imposed  simul¬ 
taneously,  and  were  random  both  in  phase  and  frequency, 
or  were  reproductions  of  the  actual  service  environment, 
the  vectors  would  resolve  properly  to  indicate  a  true 
service  life.  However,  the  application  of  a  synthetic 
environment  by  testing  In  the  three  axes  separately  does 
not  allow  proper  resolution  of  the  input  vector  forces  in 
such  a  way  os  to  provide  a  quantitative  measure  of  actual 
service  life. 

SIMULATION  OF  THE  ENVIRONMENT 


The  primary  principle  on  which  rhe  design  of  the  three 
degrees  of  freedom  test  system  rests  is  illustrated  in 
Figure  4.  This  device,  *  (called  the  Deckard  device) 
while  providing  o  high  degree  of  mechanical  stiffness  In 
the  axiol  direction  It  compliant  in  all  radial  directions. 

It  should  be  noted  that  when  a  force  it  applied  radially  at 
the  top  of  the  drive  rod,  the  drive  rad  will  translate  only, 
viz,  it  will  not  move  in  an  oicing  motion. 


Figure  3.  Electronic  Component  and  Axes 
Coordinate 


Due  to  the  configuration  of  the  support,  its  stiffness  is 
different  In  the  three  orthogonal  axes;  it  H  most  stiff  in 
the  vertical  (Y)  axis,  moderately  stiff  in  the  longitudinal 
(X)  axis,  and  softest  in  the  transverse  (Z)  axis.  The  ratio 
of  the  three  stiffnesses  might  be  of  the  order  100:10:1 
and,  since  resonance  frequencies  are  related  to  the 
square  root  of  stiffness,  the  resonance  frequencies  would 
have  rhe  ratio  of  10:3. 16:1 . 

The  deflection  of  the  body,  and,  therefore  the  stress,  is 
proportional  to  the  inverse  of  frequency  squared  and 
would  be  of  the  ratio  1:10:100  (inversely  proportional  to 
stiffness).  Fatigue,  being  a  function  of  the  product  of 
stress  and  number  of  cycles,  would  be  in  the  ratio  of 
10:31 .6:100,  assuming  that  the  inputs  in  all  three  axes 
were  of  the  same  duration  same  amplitude  ond  of  a  flat 
acceleration  versus  frequency  spectrum. 


Figure  4,  Cylindrical  Spring  Drive  Assembly 


*  Invented  by  C.E.  Deckard,  patent  applied  for. 
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Figure  5  further  Illustrates  this  concept  combined  to  pro¬ 
vide  a  three  degrees  of  freedom  motion  system.  This 
particular  concept  provides  freedom  of  motion  in  the 
three  orthogonal  axes  only,  while  providing  restraint  of 
motion  in  the  three  rotational  axes. 


f 


Figure  5,  Threc-D*gr*** -of -freedom  V*kxo»*<v» 
System 


The  t.vo  radio  sets  were  mounted  by  their  normal  bracketry 
to  the  vibration  test  system  shown  in  figures  6  and  7. 

Three  electrodynamic  exciters  were  used  to  provide  the 
three  dynamic  vibratory  inputs  to  the  system.  The  master 
tope,  containing  the  test  profile,  was  played  into  the  Con¬ 
trol  system  shown  in  Figure  8.  These  input  signals  were 
amplified  and  passed  through  filtering  networks  that  were 
tuned  to  compensate  for  fixture  and  exciter  resonances. 

A  master  control  uni*  was  incorporated  *a  provide  total 
control  of  the  entire  system. 


Figure  7.  Three-Degrees-of-Freedom  Vibration 
System 


Figure  6.  Three -Degrees -of-Freedom  Vibration 
System 


Figure  8.  Control  System  for  Three-Degrees-of- 
Freedom  Vibrotion  System 


Three  control  accelerometers  were  locoted  on  the  vibra¬ 
tion  fixture  (one  for  each  exciter).  The  output  signals 
from  the  accelerometers  were  monitored  during  testing  to 
insure  proper  vibration  levels  ond  observe  ony  test  anoma¬ 
lies.  Periodically  during  rhe  test,  the  output  signals  from 
the  control  accelerometers  and  the  tape  input  signals  to 
the  three  axis  system  were  recorded  and  analyzed  on  a 
digital  computer.  The  data  obtained  from  the  laboratory 
rests  compored  favorably  with  the  data  obtained  from  the 
flight  tests.  This  comparison  was  extremely  good  to  100 
Hz.  The  energy  recorded  at  the  higher  frequencies  was 
believed  to  be  a  result  of  acoustic  coupling  between  rhe 
Helicopter  drive  system,  the  weapons  system  (when  firing) 
and  the  radio  set.  Since  this  energy  was  not  mechanically 
coupled  through  the  aircraft  structure  and  mounting  brcc- 
kefry,  emphasis  was  placed  on  the  lower,  mechanically 
coupled,  frequencies  in  the  design  of  the  test  system. 

The  total  Grr.4  values  of  the  taped  input  and  fixture 
response  spectra  were  quite  close.  Table  I  lists  the  com¬ 
parisons  of  the  total  Gnra  levels  for  the  most  severe 
portion  of  the  simulated  profile. 
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DIVING  HUNG  PROFILE 


The  electrical  operation  of  the  two  radio  test  sets  was 
checked  at  the  beginning  and  ending  of  each  test  profile. 
This  functional  test  consisted  of  measuring  the  transmitter 
carrier  output,  the  transmitter  carrier  modulation  ond  the 
input  voltage. 

TEST  RESULTS 


-t 


where 

f(t)  =  Failure  density 
0  =  Mean  life 

t  -  Time  period  of  interest 

MTBF  is  defined  ai  the  total  test  time  divided  by  the  total 
number  of  failures.  Thus,  the  MTBF  developed  thus  for  in 
the  laboratory  is  computed  to  be: 


0 


750  ♦  750 


1500 


Since  the  MTBF  determined  experimentally  is  essentially 
only  an  estimate  of  the  true  MTBF,  statistical  confidence 
levels  must  be  placed  on  it.  The  MTBF  of  an  exponential 
failure  function  is  distributed  as  a  Chi-Square  with  2N 
degrees  of  freedom  where  N  Is  the  number  of  failures. 
There  exists  standard  reliability  tables  of  confidence 
levels  wherein  these  data  may  be  inserted  for  the  purpose 
of  determining  the  confidence  bounds.  Naturally  the 
greater  the  number  of  test  hours  imposed  on  the  tpecimtn, 
the  more  these  confidence  bounds  may  be  narrowed, 
particularly  if  more  failures  are  realized. 

At  present,  the  test  system  has  been  redesigned  for  the 
purpose  of  accommodating  four  additional  radio  sets. 
These  six  radio  sets  shall  be  subjected  to  750  hours  of 
testing.  Thus,  at  the  conclusion  of  the  program,  a 
representative,  6000  hours  of  time,  will  exist  for  MTBF 
determination, 

SUMMARY 

To  more  closely  reproduce  in  a  ground-based  laboratory, 
the  actual,  dynamic  flight  environment,  a  multi-degree 
of  freedom  test  system  should  be  used.  The  data  resulting 
from  such  o  test  system  may  be  reliably  utilized  for  the 
determination  of  the  test  item  MTBF. 


To  date,  a  total  of  775  hours  of  V'boratory  induced  vibra¬ 
tion  has  been  imposed  on  two  ra test  sets.  The  recei¬ 
ver  section  of  the  first  radio  set  malfunctioned  after  42 
hours  of  testing.  An  internal  examination  of  the  test 
specimen  showed  that  the  first  IF  stage  on  the  moin 
receiver  card  hod  detached.  It  was  necessary  to  realign 
the  RF  section  to  conform  to  standard  TM1 1-5021 -260-35. 
The  radio  set  was  repaired  by  USAAVNTSO  ond  function¬ 
ed  for  the  next  733  hours  with  no  malfunctions.  The 
second  radio  set  was  also  subjected  to  775  hours  of  labora¬ 
tory  testing  with  no  resulting  discrepancies. 

The  majority  of  reliability  criteria  assumes  the  exponen¬ 
tial  failure  rate  specified  in  MIL-STD-781B.  This  is 
expressed  by  the  equation: 


The  program  described  has  proven  the  feasibility  of  such 
a  testing  approach  by  using  commercially,  available, 
testing  equipment.  The  only  item  requiring  special  de¬ 
sign  was  the  drive  and  restraint  device  mentioned  pre¬ 
viously. 

Though  present  indications  of  the  test  data  obtained  in 
the  laboratory  seem  realistic,  final  confirmation  of  the 
test  system  validity  will  rest  upon  a  favorable  comparison 
between  the  laboratory  generated  MTBF  and  the  MTBF 
generated  in  the  field.  Should  this  comparison  be  favor¬ 
able,  it  wifi  be  possible  to  perform  future  reliability 
tests  of  this  type  of  a  greatly  reduced  cost  and  without 
the  hazards  and  schedule  difficulties  associated  with 
octual  flight  tests. 
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CHARACTERISTIC  COEFFICIENTS,  PROBABILITY  AND 
CLASSIFICATION  OF  WIND  PROFILES  (Surface  to  23  Km) 

Oskar  M.  Essenwanger 
Aerophyalca  Function 
Physical  Sciences  Directorate 
Directorate  for  Research,  Development,  Engineering 
end  Missile  Systems  Laboratory 
U.S.  Army  Missile  Command 
Redstone  Arsenal,  Alabama 

ABSTRACT .  An  attempt  is  made  to  derive  «  limited  number  of  realistic  wind 
profiles  on  a  global  basis  under  consideration  of  probability  thresholds  for 
analysis  of  the  environmental  influence  upon  missiles. 

In  the  past,  synthetic  wind  profiles  were  available  for  certain  selected 
probability  levels.  They  do  not  reflect  the  true  altitude  relationship  of  the 
wind  profile  and  association  with  probability  la  selective  and  ambiguous. 
Individual  wind  profiles  or  correlation  matrices  overcome  problems  of  unrea¬ 
listic  description  but  input  into  computer  programs  Is  generally  bulky  and 
computer  runs  may  be  very  costly  and  time  consuming.  Random  selection  of  indi¬ 
vidual  wind  profiles  limited  in  number  introduces  bias  by  persistence  and/or 
gaps  in  present  meteorological  upper  air  data  collection. 

A  new  technique  developed  by  the  author  overcomes  the  above  cited  short¬ 
comings.  The  Individual  wind  profile  is  approximated  by  a  number  of  characteris¬ 
tic  coefficients,  which  can  be  reduced  to  one  variable.  The  global  condition 
can  now  be  deacribed  by  three  equations  and  a  set  of  constants.  This  serves  ss 
the  basis  to  classify  wind  profiles  into  typical  families. 

Five  major  groups  whose  subtypes  depend  on  the  desired  accuracy  end  the 
particular  missile  problem  to  be  solved  have  been  found  sufficient.  In  our  case 
1A  subgroups  have  been  established  with  a  total  of  49  wind  profile  models.  For 
many  first  survey  goals  it  la  only  necessary  to  analyse  the  effect  of  5  -  6  wind 
profiles. 

The  seasonal  and  geographic  variation  of  the  groups  is  discussed. 

INTRODUCTION.  The  establishment  of  appropriate  wind  profiles  for  design 
and  trajectory  analysis  has  always  been  an  intricate  problem  in  missile  clima¬ 
tology.  It  is  difficult  to  derive  proper  wind  profiles  with  inclusion  of  the 
true  vertical  relationship  and  assessment  of  the  proper  probability  thresholds, 
and  not  to  end  up  with  bulky  Inputs  of  data  for  computations  in  the  analysis 
of  the  environmental  influence  upon  missiles. 

In  the  past  several  solutions  have  been  recommended.  As  a  trivial  approach, 
individual  wind  profiles  have  been  employed.  Let  us  assume  key  stations 
reflecting  global  representations  can  be  selected;  this  met  tod  has  solved  the 
problem  of  true  vertical  relationship.  Remaining  is  the  voluminous  input  into 
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computer  programs  which  can  cause  storage  problems,  and  the  analysis  may  be 
very  costly  and  suffer  from  inhomogeneity  and  data  bias,  as  upper  air  obser¬ 
vations  decrease  considerably  in  number  with  altitude,  and  data  gape  exist. 

If  only  a  limited  number  of  profiles  were  selected  (even  by  Monte  Carlo  methods) 
the  results  would  leek  completeness  and  would  not  be  conclusive. 

Data  Input  is  curtailed  to  some  degtee  by  the  establishment  of  intra  and 
interl’vel  correlation  matrices  (i.e.  Court,  1957).  Theit  calculation,  however, 

Is  cosvly  and  complex,  and  probability  thresholds  cannot  readily  be  derived. 
Virtually  little  Is  gained  over  the  previous  method  to  simplify  design  analysis. 

The  third  technique  is  based  upon  limited  nv  -  of  mostly  synthetic  wind 
profiles  (i.e.  Slssenwlne,  1954;  Handbook  of  Goo;  •  a  and  Space  Environment, 
etc.).  This  cues  down  on  analysis  cost,  but  in  mauy  of  these  synthetic  profiles 
the  true  vertical  relationship  is  neglected.  Further,  profiles  must  be 
established  for  any  number  of  thresholds,  association  wltn  wind  shear,  gust,  and 
turbulence  is  difficult,  and  a  true  probability  is  hard  to  assess.  Modern  tools 
of  statistical  analysis  such  as  a  Monte  Carlo  method  could  not  be  applied. 

A  new  approach  has  been  suggested  by  the  author.  The  individual  wind  profile 
is  mathematically  described  by  a  number  of  coefficients.  Their  interrelation¬ 
ship  wa '  studied  and  it  proved  that  Interdependence  permits  reduction  to  one 
characteristic  coefficient.  The  global  wind  conditions  can  now  be  described  by 
three  equations  and  a  set  of  constants  varying  by  month.  One  of  the  equations, 
the  frequency  distribution  of  the  characteristic  coefficient,  constitutes  the 
link  between  wind  profile  and  probability  of  occurrence.  With  it  any  selected 
threshold  can  be  studied  without  modification  of  the  input  which  is  minimised 
to  a  fraction  of  less  then  one  thousandth  of  the  original  data. 

The  new  technique  serves  as  the  bacis  for  classification  of  the  wind 
profile  into  essentially  five  typical  families,  whose  submodels  depend  on  the 
accuracy  tad  the  individual  missile  problem  to  be  solved.  A  quick  survey  of 
the  wind  influence  upon  missile  systems  would  necessitate  a  maximum  of  14  profile 
models  for  detailed  evaluation,  but  six  extreme  profile  models  would  suffice 
as  a  first  ajpraoch.  Since  probability  of  the  models  on  a  global  basis  la 
known,  the  proper  assessment  of  the  wind  influence  upon  the  missile  system 
under  study  can  be  cede. 

The  technique  may  have  potential  for  other  design  and  analysis  problems^ 
and  derails  mav  be  Ditaented  in  the  following  that  other  applications  can 
be  wade. 


THE  DEVELOPMENT  OF  WIND  MODELS 


«.  Mathematical  Representation  of  the  Wind  Profile 

In  previous  reports  (Essenwanger,  1964  and  1970,  Essenwanger  and 
Boyd,  1970)  the  mathematical  representation  of  the  wind  profile  has  been 
treated.  It  was  pointed  out  that  the  wind  speed  profile  from  surface  to  about 
25  km  can  be  best  described  by  terms  of  a  Fourier  Series,  while  the  wind  direc¬ 
tion  profile  is  more  readily  approximated  by  (orthogonal)  polynomials.  Although 
other  forma,  l.e.  expressing  the  wind  profile  by  mathematical  terms  for  the 
zonal  and  meridional  components,  could  be  employed  (see  Essenwanger,  1964), 
the  wind  speed  and  direction  system  was  adopted  In  the  following  as  best  suited 
for  Input  into  missile  design. 


The  interrelationship  between  coefficients  was  studied  by  Essenwanger 
and  Boyd  (1970),  and  the  conclusion  was  drawn  that  only  one  character istic 
coefficient  Is  necessary  to  represent  the  wind  speed  profile  V  with  sufficient 

accuracy.  This  provides  the  following  equation  for  a  wind  profile  system  from 
surface  through  25  km 

V.  -  V.  •  (A  -  A  )[1  +  k.  sin  (a.  +  a,)  +  k  sin  (2a  +  a  ).+  k  sin  (Ja  t  a  )J 

hhoo  1  hl2  hzo  no 

(1) 


where  h  denotes  the  altitude  dependency,  H  the  top  altitude  and 

2  *h  360h  .  „  .  ... 

“h  "  IT  "  ”h —  •  b  -  0,  1,  ...  (H  -  1)  (la) 

with  V  -  A  (1  +  k  sin  (a  ♦  a. )  +  ...).  (lb) 

no  i  n  i 

The  Aq,  k^,  k2,  k^r  a^t  a^,  stand  for  constants  varying  with  season  and 
geographic  location  for  sets  of  individual  wind  profiles. 


The  a^,  although  different  at  Individual  altitude  levels,  assumes  the  sate 

value  for  every  profile  and  for  an  individual  time  Interval  or  particular  loca¬ 
tion,  end  the  Aq  is  the  only  variable  parameter  left,  Therefore  the  frequency 

distribution  of  A  takes  on  the  role  of  a  probability  scale  for  wind  profiles, 
o 

and  any  dealred  threshold  of  exceedance  of  design  criteria  can  chug  be  established 
from  the  frequency  distribution  of  A  .  As  a  suitable  form  the  Weibull  distri¬ 
bution  has  been  chosen, although  other  curves  may  fit,  too.  This  provides  the 
equation  for  the  cumculative  distribution. „ 


F(x) 


(2) 
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where  the  x  is  identical  with  the  A^  and  the  y,  6,  S  are  parameters  of  the 

distribution,  Estimation  of  the  parameters  can  be  made  by  various  methods 
(see  E3senwanger,  1968  and  1971), 

The  profile  for  any  desired  threshold  can  be  obtained  by  calculating 
Aq  for  this  threshold.  This  is  equivalent  to  determining 

*th  -  Y  +  61&1  1/(1  -  Fx))1/S  (2a) 

and  then  computing  the  profile  by  equation  (1)  with  this  threshold  value.  The 
F(x)  denotes  the  probability  of  the  specified  threshold. 

The  third  equation  relates  the  wind  direction  b  to  altitude  and  Aq.  Since 

the  correlation  between  Aq  and  the  wind  direction  profile  is  almost  non  existent, 

a  mean  wind  direction  profile  can  be  associated  with  the  above  system.  This 
leads  to 


6h  ‘  *w  +  Vlh  +  c2*2h  +  **•  cS*Sh  (3) 

where  ...  are  orthogonal  (Tche'oychef f )  polynomials  (see  Essenwanger,  1964 

and  1970) ,  the  Is  the  mean  wind  direction  profile  by  altitude  and 

Cj,  Cj  ...  Cj  vary  with  season  and  location  commensurate  with  the  constants 

of  equation  (1).  It  should  be  noticed  that  the  parameter  b  of  equation  (2) 
and  of  equation  (3)  are  not  identical. 

The  three  equations  satisfy  the  postulated  conditions:  a  realistic  description 
of  the  true  wind  profile  under  inclusion  of  the  vertical  relationship,  an 
association  with  probability  of  occurrence  ans  a  limitation  In  the  numbers  of 
lnpuc  quantities. 

The  method  has  been  successfully  applied  to  evaluation  of  the  wind  influence 
In  the  development  of  new  missile  systems. 

b •  Concepts  for  Deriving  Sets  ot  Wind  Models 

The  mathematical  description  serves  as  the  basis  for  the  development 
of  a  global  set  of  wind  models  which  can  be  utilised  in  analysis  and  tactical 
operation  of  missile  systems.  The  extreme  models  of  the  set  would  be  available 
for  quick  analysis  and  simple  studies  of  design  and  evaluation  of  wind  environ¬ 
ment  upon  missile  concepts. 
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The  derivation  of  a  eet  of  wind  models  la  generally  a  subjsctivs  teak 
depending  on  accuracy  requlremant,  practical  application  and  limitations  set 
by  the  configuration  of  a  missile  system.  Thus  the  number  of  models  will  vary 
with  the  posted  goals.  The  technique  of  derivation  is  general,  however,  and 
will  subsequently  be  discussed.  These  techniques  may  have  potential  application 
beyond  the  narrow  field  of  wind  analysis. 

It  has  been  demonstrated  that  equation  (1)  contains  constants  varying  with 
season  and  location  and  thus  s  global  combination  with  ons  frequency  distribution 
end  e  global  sat  of  constants  Is  not  advisable,  although  formalistic  computation 
of  constants  for  this  global  set  would  be  possible.  This  can  also  be  Inter¬ 
preted  that  the  complexity  of  environmental  wind  conditions  cannot  be  reduced 
to  one  simple  system.  Thus  the  trivial  solution  of  wind  modsls  by  class  intervals 
of  Aq  on  a  global  basis  la  not  very  promising. 

It  would  be  possible  to  derive  a  set  of  models  for  every  month  at  specified 
representative  locations.  These  models  could  be  combined  into  a  global  set. 
Duplicate  models  could  be  eliminated,  thus  reducing  the  number  of  models.  The 
variety  of  conditions  end  the  existence  of  models  with  very  low  probability 
of  occurrence  make  this  method  little  attractive. 


Analysis  of  equation  (1)  leads  to  che  conclusion  that  the  wind  speed  profile 
is  a  mixture  of  vaves,  in  which  the  constants  k.,  k,  and  k.  are  the  key  to  the 
mixture  or  the  weight  of  the  individual  waves,  This  can  be  quickly  Illustrated 
us  for  the  system 


This  scheme  Is  free  of  A  and  depends  on  che  constants  k,  only.  It  may  therefore 

O  X 

be  possible  to  build  a  set  of  models  from  class  divisions  of  wave  dominance  or 
mixture,  A  similar  technique  has  been  utilised  for  sound  speed  profiles  where 
the  vertical  structure  of  the  profile  was  the  deciding  factor  for  appearance  of 
acoustic  focusing  (see  Essenwanger,  1966). 

In  an  earlier  Investigation  (Essenwanger,  1970b)  the  mixture  of  waves  was 
based  on  monthly  mean  conditions,  and  model  sets  were  derived  from  there.  Since 
some  individual  profiles  were  poorly  classified,  however,  a  new  scheme  was  tried 
with  individual  profiles. 
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First,  a  schematic  objective  classification  was  set  up  with  criteria  ilka 
100%  of  a  single  wave  (k,  *  1.0,  k.  »  k,  •  0),  a  double  wave,  triple  wave, 

1  4  j 

mixture  of  waves,  etc,,  and  a  first  frequency  count  of  the  individual  profiles 
falling  Into  these  classes  followed.  As  expected,  many  of  the  classes  remained 
empty,  although  a  total  of  four  representative  stations  with  about  18,000  profiles 
was  employed.  Then  the  number  of  conditions  was  reduced  until  finally  only 
eight  typical  groups  (families)  remained.  It  was  recognised  from  analysis  of  the 
individual  profile  fitting  thst  the  wind  models  had  to  be  based  on  e  slightly 
modified  form  of  (1)  by  subtracting  (lb)  and  introducing  a  factor  B 

V.  “A  (1  +  Blk.  sin  (a  +  a  )  +  k,  sin  (2a.  +  a  )  +  k,  sin  (3a,  +  a  )]}  (5) 

no  1  h  1  2  h  2  3  !i3 

The  B  serves  as  an  adjustment  between  A  and  k.,  Although  the  percentage 

O  1 

reduction  is  not  changed,  and  with  it  the  mixture  of  waves  (harmonic)  remains 
the  same,  this  permits  a  shift  of  the  mean  value  A  (profile  reference).  In  the 
prior  scheme  (Essenwanger ,  1970b)  based  on  the  monthly  mean,  the  B  assumes  the 
value  1.0.  Thus  some  Individual  profiles  displayed  systematic  deviations  from 
the  best  fitting  prototype.  This  deficiency  needed  correction. 

The  factor  B  •  A^/Aq  was  computed  for  all  individual  profiles  and  a  frequency 

distribution  established.  (The  A^  Is  the  coefficient  of  the  first  harmonic,  see 

Essenwanger  and  Boyd,  1970).  Although  B  varied  from  0.4  to  1.4,  the  mode  was 
1.0.  Significant  deviations  from  the  regular  models  appeared  for  B  2.  1.3  and 
B  _<  0.6.  There  were  only  a  few  cases  out  of  the  18,000  profiles  with  B  1.3. 

They  could  readily  be  accommodated  into  other  prototypes  of  models.  Thus  the 
condition  was  cancelled  and  only  B  _<  0,6  was  kept.  It  proved  that  only  two 
families  of  models  needed  this  modification. 

The  phase  angle  was  added  from  .he  frequency  distribution  within  the  model 
groups.  Although  numerous  combinations  are  possible,  only  a  few  significant 
prototypes  emerged.  Elimination  of  non-extreme  rare  profile  types  close  to 
other  models  resulted  in  the  scheme  of  Table  1  with  48  individual  wind  speed 
profile  models.  A  model  A  •  0  as  number  49  was  added.  Since  the  closest  models 
next  to  It  employed  an  amplitude  Aq  of  6  m/sec,  only  IX  of  Che  individual  profiles 

occupied  class  49.  This  would  be  different,  If  other  spacing  between  models 
Were  chosen  than  the  one  illustrated  In  Table  1,  i.e.,  the  first  amplitude  would 
be  8  m/sec.  Thus  49  rounds  out  the  set  of  models. 

Table  1  displays  the  final  arrangement  of  models.  Only  five  main  groups 
(families)  were  left  with  varying  subtypes.  Within  a  subtype  several  models 
have  been  derived  by  classes  of  A^,  Although  in  the  beginning  more  subgroups 

with  varying  a^  had  been  established,  occupancy  of  the  model  classes  by  observed 

models,  proximity  to  other  models  and  consistency  within  families  reduced  the 
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Table  1 


Configuration  of  Models  based  on  Equation  ( 5) 


number  Co  the  48  typea  shown  in  Table  1.  The  models  with  B  i  1.0  war*  given  the 
coda  index  sero  in  the  second  digit.  Others  were  then  numbered  as  displayed  in 
Table  1.  More  details  on  the  goup  of  wind  models  will  be  discussed  in  the 
next  section. 

c.  The  Families  of  Wind  Models 


Table  1  summarises  the  respective  constants  utilised  in  the  computation 
of  the  wind  speed  models.  It  should  be  noticed  that  only  five  main  conditions 
remained  from  a  variety  of  initially  postulated  profile  mixtures.  This  does  not 
mean  that  other  groups  would  not  exist.  It  was  decided,  however,  that  all 
groups  which  placed  less  than  five  observed  profiles  out  of  the  18,000  studied 
were  considered  negligible. 

Within  the  group  several  subgroups  were  derived  by  alteration  of  the  a 
(phase  angle).  Within  the  subgroups  models  were  established  with  the  spacing  of 
the  Aq  as  given  in  the  headings  of  Table  1.  Originally  equal  spacing  of  A 

waa  considered.  Studies  of  the  proximity  of  models,  frequency  of  occurrence 
and  accuracy  of  the  distributed  observed  profiles  led  to  the  present  setting. 

Equal  spacing  of  A^  would  have  resulted  in  at  least  10  to  1?  more  models  with 
virtually  no  gain  °ln  accuracy. 

The  first  group  contains  profiles  with  a  single  maximum,  while  the  second 
group  exhibits  a  double  maximum  profile.  The  th<rd  group  Is  a  mixture  between 
the  first  and  second  harmonic.  Following  is  a  group  with  a  main  minimum,  as 
the  single  wave  dominates  in  this  mixture.  The  last  group  is  an  equally  balanced 
combination  of  the  first  three  harmonics. 

2 

Although  the  I  k^  for  this  last  group  would  result  in  a  sum  of  0.75,  while 

all  other  groups  add  up  to  1.0,  this  deviation  is  intentional.  It  can  be  readily 
seen  by  equation  (4a)  that  the  change  of  the  sum  does  not  influence  the  percentage 
reduction.  The  modification  determines  spacing  of  the  models  within  the  group. 

A  study  of  Che  closeness  of  fit  within  the  individual  groups  revealed  that  the 
largest  deviations  between  observed  and  analytic  profile  were  found  In  thfs  last 
group.  The  selection  of  k  as  given  by  Table  1  reduced  the  spacing  of  the  models 
and  made  it  commensurable  with  the  distance  in  the  other  groups. 

Within  a  group  initially  all  models  as  obtained  from  the  spacing  of  A^ 

were  computed  and  run  through  a  comparison  program.  The  mean  squared  difference 

2  *)  2 
c  between  models  was  computed,  and  all  models  with  c  <  16  were  eliminated. 

Although  these  wind  speed  profile  models  would  certainly  exist,  proximity  to 

ocher  prototypes  make  them  surplus.  When  there  was  a  choice  between  models,  the 

one  with  the  lower  subgroup  number  was  kept.  Thus  various  models  with  amplitude 

A  •  6.0  were  omitted. 
o 

*) 

See  Equation  (6). 
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Figure  1  Illustrates  the  model  types  tn  groups  11  and  12.  They  differ 
by  the  altitude  of  the  maximum.  In  group  11  the  maximum  appears  at  14  -  16  km, 
a  property  found  in  the  tropics  (see  Table  3),  while  group  12  shows  a  maximum 
around  10  -  12  km,  expected  In  subtropical  and  midlatitude  regions. 


Figure  2  serves  for  comparison  of  the  groups  10,  11,  and  12,  The  series 
of  models  8  through  13.  although  similar  to  group  12,  Includes  a  displacement 
of  the  profile  reference  Afl,  which  makes  some  of  the  observed  profiles  better 
than  the  series  5  through  7. 


Figure  3  Illustrates  the  profile  types  with  double  and  triple  waves.  The 
extreme  profile  models  were  chosen  from  each  subgroup  to  demonstrate  the  indi¬ 
vidual  features  better  than  models  with  smaller  amplitude  A  .  Again,  the 
difference  between  subgroups  21  and  22  Is  the  altitude  of  tfte  maximum  and  the 
minimum  wind  speed.  Profile  subgroup  21  displays  a  shift  towards  higher  altitude. 

The  difference  In  phase  angle  for  subgroups  50,  51,  and  52  causes  the  side- 
maxims  in  the  troposphere  and  lower  stratosphere  to  vary  In  strength  compared 
with  on^  another,  while  the  main  maximum  speed  stays  above  20  km. 

Figure  4  finally  exhibits  a  dominant  single  wave  (groups  3  and  4),  The 
major  difference  from  the  single  wave  pattern  subgroups  10  through  12  is  the 
narrowlt-g  of  the  maximum.  Some  shift  in  altitude  for  the  maximum  of  the  Indi¬ 
vidual  subgroups  is  visible,  too,  as  should  be  expected  from  the  phase  angles 
given  in  Table  1. 

The  geographic  and  seasonal  distribution  follows  In  section  2d.  Before 
detailed  consideration  of  time  and  space  variability  a  short  look  at  the  close¬ 
ness  of  fit  may  be  appropriate. 


Since  Che  spacing  of  the  models  Is  an  arbitrary  decision  of  selecting  a 
few  from  the  numerous  possible  sequence  within  one  family,  6ome  objective 
criteria  should  be  derived  for  indication  of  the  quality  of  the  model  sets. 
This  can  be  accomplished  by  computing 


2 

e 


£  1  <*,, 


y  )2]1/2 

yhk'  ‘ 


(6) 


where  the  h  denotes  the  aldtude  and  j  and  k  two  different  models,  one  of  these 
can  be  the  observed  profile.  The  mean  squared  difference  can  also  be  employed 
to  place  the  observed  profile  into  one  model  type.  We  select  the  closest  to 
the  observed  speed  profile,  l.e,,  the  minimum  e‘.  This  Is  the  same  procedure 
by  which  the  observations  were  associated  with  the  former  set  of  models 
(Essenwanger ,  1970b).  Comparison  with  these  43  models  derived  from  monthly 
average  conditions  shows  that  the  present  Bet  of  models  fits  the  individual 
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prof list  closer,  although  only  six  more  models  were  necessary, 
discerns  the  distribution  of  che  c ^  by  station  and  season.  The 
takas  place  in  the  subtropical  tone  (Montgomery)  and  In  winter, 
parallels  the  conclusion  from  the  43  model  set.  The  minimum  e* 
and  season  emerges  for  Albrook  and  the  surater. 

d.  Geographic  and  Seasonal  Variation  of  Models 

The  discussion  on  sets  of  wind  models  would  be  Incomplete  without  the 
analysis  of  the  time  and  space  variability  of  the  models.  A  summery  for  the 
four  stations  is  given  in  Table  3,  the  seasonal  variation  by  station  in  Table  4. 
Since  the  number  of  observations  varied  by  station  and  season,  (see  Table  4) 
a  conversion  to  0.1X  was  made  and  the  station  summary  and  global  combination 
was  based  on  the  homogenised  number.  This  would  not  eliminate  the  bias  intro¬ 
duced  by  data  gaps  within  the  seasons  or  the  period  of  record  utilised,  but 
balances  the  disproportionate  weight  of  summer  data  within  the  year  and  the  four 
dally  observations  at  Thule  compared  with  two  daily  recordings  at  the  other 
stations  in  the  global  summary. 

The  last  column  in  Table  3  summarizes  the  global  conditions.  No  subgroup 
contains  less  chan  IX.  The  only  frequency  less  than  IX  is  the  profile  model 
49,  (Aq  -  0)  which  can  be  considered  calm  air  for  practical  purposes.  Expla¬ 
nation  was  given  In  the  preceedlng  section  that  the  number  of  profiles  plnced 
into  this  model  type  depends  on  the  spacing  of  the  next  A  .  The  type  is 
necessary  as  a  boundary  co  complete  the  series  of  models. 

The  most  frequent  subgroup  is  10,  a  profile  with  a  single  maximum  at 
10  -  12  km.  As  expected,  this  is  a  midlatitude  type  and  comprises  about  SOX 
of  the  wind  profiles  at  Chateauroux.  It  is  also  the  type  with  the  maximum 
frequency  ct  Thule  and  Montgomery  and  the  moat  frequent  one  in  all  seasons. 

The  next  subgroup  in  number  of  placed  profiles  is  50,  a  mixture  of  all 
three  harmonics  with  the  major  maximum  above  20  km  (Figure  3).  It  should  be 

noticed  that  the  decrease  in  wind  speed  towards  25  km  indicated  in  Figure  3 

la  a  result  of  the  mathematical  representation  of  the  profile  by  three  terms 
only  and  the  selection  of  the  altitude  ranee  from  surface  through  25  km.  In 
moat  empirical  profiles  this  decrease  of  the  wind  speed  around  25  km  cannot  be 
observed  and  the  maximum  actually  lies  at  a  higher  level  than  25  km.  The 
limitation  in  the  mathematical  description  produces  the  maximum  around  23  km. 

The  importance  is,  however,  that  thiB  subgroup  displays  an  Increase  of  the  wind 
speed  In  the  altitude  range  between  20  to  25  km  exceeding  the  maximum  speed  at 
the  tropospheric  level  (Jet  stream).  This  type  of  wind  profile  is  typical  for 

the  polar  and  tropical  region  as  can  be  noticed  from  the  almost  25X  frequency 

at  Albrook  and  Thule.  Table  4,  giving  the  seasonal  breakdown  by  station, 
exhibits  a  dominance  of  53X  at  Thule  in  winter,  while  at  Albrook  this  subgroup 
persists  in  summer  with  36X.  It  explains  the  high  occurrence  for  the  year  at 
both  stations.  The  peak  season  at  Thule  and  Albrook  is  opposite,  however. 


Table  2 

highest  dispersion 
This  result 
per  stetion 
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Average  Squared  Deviation  Between 
Wind  Model  and  Observed  Profile 

(  In  0.1O 


e 

Al 

Mo 

Cha 

Thu 

Wl 

Sp 

Su 

Fa 

*  i 

1 

1 

1 

1 

1 

l 

2-4 

18 

10 

35 

46 

10 

3: 

35 

30 

5  -  9 

140 

90 

164 

205 

66 

157 

200 

177 

10  -  16 

309 

244 

275 

280 

20 

97 

312 

"97 

17  -  25 

363 

31C 

267 

221 

28.2 

285 

295 

279 

26  -  36 

162 

184 

13C 

118 

192 

136 

121 

144 

37  -  ^9 

26 

86 

71* 

57 

110 

56 

27 

5C 

50  -  64 

2 

36 

32 

29 

59 

19 

7 

14 

65  *  8l 

24 

12 

20 

40 

10 

l 

5 

9 

5 

8 

16 

3 

1 

2 

>  100 

7 

5 

15 

22 

1 

*♦ 

1 

Average 

4.30 

5.01 

4.62 

4.63 

5>7 

4.49 

fc.15 

4.37 

Al  •  Albrook  (Canal  Zone) 
Mo  ■  Montgomery  (  Alabama) 
Cha  -  Chateauroux  ( France) 
Thu  ■  Thule  (Greenland) 

Wl  ■  December  -  February 
Sp  ■  March  -  May 
Su  «  June  •  August 
Fa  -  September  -  November 
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Table  4 


SEASONAL  VARIATION  OF  WIND  PROFILE  GROUPS  FREQUENCY  (  In  O.i*) 


Model 

Albrook 

j  Montgomery 

|  Chateautoux 

Thule 

Group 

Wi 

sp 

Su 

Fa 

Wl 

Sp 

Su 

Fa 

Wl 

Sp 

s«< 

Fa 

Wl 

Sp 

Su 

Fa 

10 
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266 

3  7 
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550 

504 
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90 
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i  76 

11 
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157 

21 

51 
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66 

17 

16 

6 

13 

37 

3 

34 

12 

- 

- 

- 

- 
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3 

94 

10 

58 

82 

43 

- 
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* 

- 

21 

9U 

.In 

85 

148 

0 

336 

37 

50 

71 

36 

35 

47 

44 

23 

7- 

-- 

7 
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35 

C 
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4 

0 

1 

3 

15 

2; 

2 

6 

51 

9 

0 

0 

0 
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t- 

14 

54 

26 

67 
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66 

64 
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7 

9 

45 

- 

- 

- 

- 

c 

1 

0 

C 

47 

58 

15 

On 
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46 

80 

67 

4i 

_ 

. 

_ 

_ 

9 

18 

6 

■'■0 

51 
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ICO 

87 

47 

48 

77 

<7 
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1 

0 

0 

11 

62 
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29 
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10 

16 
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48 

- 

- 

- 

- 

•+5 
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1 

0 

1  1 

709 

162 

52 

160 

3 

f. 

1 

14 

- 

- 

- 

- 

45 

13 

34 

8 

1  *• 

10 

43 

74 

67 

78 

43 

55 

105 

346 
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703 

-  "\ 

178 

166 

3  64 

194 

8 

c 

133 

8 

•59 

70 

1 

25 

5:8 

14b 

0 

1 

.1 

59 

17 

9 

9 

70 

0 

4l 

4 

80 

17 

0 

11 

38 

16 

0 

54 

107 

109 

125 

185 

C 

0 

5 

1 

17 

4 

0 

0 

58 

8 

0 

6 

Calm 

0 

0 

0 

p 

0 

0 

0 

0 

0 

3 

4 

6 

0 

30 

56 

9 

Weak 

i 

; 

686 

4°5 

590 

11 

158 

613 

293 

175 

347 

33C 

356 

568 

845 

457 

N 

1 

:  r-s 

951 

870 

854 
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84u  IO85 
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8  1 

1031 

983 

916 

599 

1951 

2535  1774 
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Other  subgroups  show  less  than  10X  In  the  total  summary.  Noticeable  la 
the  25X  of  subgroup  45  at  Thule,  which  is  a  profile  type  with  predominance  of 
a  single  wave  but  a  narrow  altitude  region  for  the  maximum.  The  altitude  of 
the  maximum  is  rather  low  (Fig.  4,  profile  38).  The  subgroup  has  its  peak 
frequency  in  spring  and  summer. 

Since  the  first  amplitude  Is  6  m/aec  only,  profile  types  in  all  subgroups 
comprise  situations  when  the  wind  speed  la  not  very  strong.  They  have  been 
combined  under  "weak"  wind  profiles  irrespective  of  the  subgroups  and  the  model 
49  has  been  included.  It  Is  noteworthy  that  Albrook  and  Thule  emerge  aa  stations 
where  about  SOX  of  the  wind  profiles  are  placed  Into  these  weak  wind  speed 
profile  groups.  As  expected,  the  subtropical  and  midlatitude  regions  have  by 
far  stronger  winds.  The  seasonal  change  should  also  be  pointed  out,  which 
varies  between  about  60X  in  summer  and  20X  .'n  winter.  Although  this  result  Ib 
a  well  known  fact,  the  confirmation  In  objective  numbers  is  satisfying,  and  it 
demonstrates  that  the  system  of  wind  profile  models  is  reasonably  rational. 

The  season  of  the  peak  for  the  number  of  weak  wind  profiles  at  Albrook,  the 
spring,  may  be  a  surprise.  Somewhat  unexpected  may  also  be  the  low  IX  In  winter 
at  Montgomery.  The  high  85k  In  summei  in  Thule  is  striking.  Otherwise  the 
seasonal  and  station  distribution  is  as  anticipated. 

Table  5  summarizes  the  distribution  by  combining  the  subgroups  10  -  12 
and  40  -  45  into  one  group  of  single  maximum  profiles  and  lists  each  of  the 
other  three  main  groups.  Since  the  "calm"  model  has  been  omitted,  the  percentage 
figures  do  not  add  up  to  100X  for  all  seasons  or  situations. 

Table  5  reveals  that  the  single  maximum  profile  type  is  dominant  in  the 
midlatitudes  and  subtropics.  Montgomery  shows  more  than  90X  of  this  group  in 
all  hut  the  summer  season,  while  only  50X  of  this  group  is  left  for  the  winter 
season  in  Chateauroux.  This  agrees  with  the  expected  seasonal  shift  of  the 
Jetstream . 

The  tropical  region  (Albrook)  appears  with  a  split  between  chree  major 
groups.  The  peak  frequency  shifts  from  a  single  wave  structure  In  winter 
with  a  maximum  around  the  tropopause  to  the  group  with  a  maximum  in  the 
stratosphere  in  summer.  The  reverse  change  takes  place  in  the  polar  region 
(Thule) . 

The  double  wave  pattern  (21  -  22)  seems  important  only  in  the  tropical  zone 
during  all  seasons  and  In  the  subtropical  regime  during  summer.  The  group 
31  -  32,  a  mixture  between  single  and  double  wave,  appears  significant  In  the 
midlatitude  ar.d  polar  region. 
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Suninary  of  Profile  Group*  ( In  /o) 
Seaaor.al  Variation 


Model 

Group 

W1 

Albrook 
Sp  Su 

Fa 

Montgomery 
Wi  Sp  Su 

Fa 

Chateauroux 

Wi  Sp  Su  Fa 

Single 

55 

44 

20 

U5 

96 

98 

92 

51 

82 

88 

83 

21  -  22 

10 

27 

22 

18 

0 

1 

3^ 

4 

5 

2 

4 

4 

51  -  V 

1 

0 

0 

0 

1 

\ 

5 

3 

11 

1 

8 

9 

50  -  52 

34 

29 

50 

59 

5 

0 

18 

1 

35 

1* 

0 

3 

Station  and  Season  Survey 


Model 

Group 

Al 

Stations 

Mo  Ch 

Th 

Wi 

Seasons 

Sp  Su 

Fa 

Single 

42 

6,' 

76 

58 

57 

73 

60 

68 

21  -  22 

19 

10 

4 

6 

5 

9 

16 

9 

31  -  32 

). 

1 

10 

7 

5 

5 

6 

5 

50  -  52 

38 

5 

10 

27 

33 

12 

17 

16 
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The  predominance  of  one  group  at  most  of  the  stations  during  the  seasons 
makes  it  plausible  thac  the  mathematical  representation  of  the  wind  speed 
profile  and  the  subsequent  reduction  to  one  coefficient  (Easenwanger  and  Boyd , 

1970)  succeeds  for  monthly  summaries.  It  can  be  recognised,  however,  that  a 
global  combination  of  the  frequencies  of  one  parameter  will  not  render  a  good 
representation  of  the  structure  of  the  wind  profile,  as  the  average  constants 
for  a  global  combination  would  not  express  the  variety  of  the  profile  groups. 

SUMMARY  AND  CONCLUSIONS.  The  mathematical  description  of  the  wind  profile 
by  one  variable  characteristic  coefficient  and  monthly  nets  of  constants  by 
location  (Essenwanger  and  Boyd,  1970)  have  served  as  the  foundation  for  the 
derivation  of  a  global  set  of  wind  speed  profile  models.  Groups  of  models  are 
established  by  utilizing  the  percentage  reduction  (equation  4  and  4a)  of  three 
coefficients.  Although  numerous  class  groups  could  have  been  established, 
about  18,000  profiles  from  four  representative  stations  of  four  climatic  zones 
were  divided  into  essentially  five  major  typical  families  of  profiles  with  a 
total  of  14  subgroups.  The  subgroups  and  the  subsequent  Individual  prototypes 
of  profiles  depend  on  accuracy  and  individual  goals  for  missile  systems. 

The  presented  groups  divide  into  49  wind  profile  models,  whose  14  extreme 
profiles  of  the  lndividusl  subgroups  permit  quick  calculation  of  configuration 
for  missile  systems  and  evaluation  of  problem  areas.  The  14  extreme  profiles 
could  be  further  reduced  to  essentially  5-6  wind  speed  profiles,  representing 
che  major  groups.  Since  the  global  probability  of  occurrence  for  the  individual 
groups  la  known  (Tables  3;  4,  5),  evaluation  of  the  wind  Influence  upon  missile 
systems  or  in  trajectory  analysis,  etc.,  should  be  possible  with  relatively 
little  coat. 

The  developed  set  of  49  profiles  can  also  be  employed  in  tactical  operations 
or  other  systems  of  operational  purpose,  The  set  of  49  models  fits  the  indi¬ 
vidual  wind  speed  profiles  better  chan  an  earlier  set  of  43  models  derived 
from  mean  monthly  conditions.  The  set  of  49  models  serves  further  to  investigate 
che  association  with  other  wind  parameters  such  as  wind  shear,  turbulence,  gust, 
etc.,  and  association  with  density  and  temperature  profiles.  A  further  application 
lies  in  the  study  of  predictability  of  the  models  for  tactical  purposes. 

Although  the  49  models  presented  here  are  synthetic  profiles,  their  origin 
from  the  description  of  the  vertical  structure  of  the  wind  speed  profile  makes 
them  a  realistic  approximation  of  the  true  conditions.  Therefore  system  analysis 
with  -hem  should  lead  to  maximum  efficiency  end  accuracy  with  minimum  cost  to 
missile  systems,  avoiding  over  -  or  under  design.  The  problem  of  voluminous 
data  input  in  studies  of  the  environmental  effect  is  solved  and  data  bias  and 
inhomogeneity  is  largely  reduced. 


The  seasonal  and  geographic  variation  of  the  prof  lit  group*  ahova  excellent 
agreement  with  known  lntaractlon  In  the  general  circulation.  This  aupporta  the 
face  that  the  wind  models  are  capable  of  expressing  the  true  atructura  of  th* 
atnoaphara.  A  aophiatlcatad  aodlflcatlon  of  the  models  would  ha  the  correction 
of  a oca  daflclenciea  of  the  three  term  harmonic  recoaputatlon  by  comperlaon 
with  tha  empirical  profile*  placed  lneo  the  prototype  dlvlalon. 

The  technique  of  deriving  model*  from  a  aat  of  global  descrlptora  or  a 
ccuntlaaa  number  of  obcervatlona  haa  general  application  in  atatlatical  analyala 
and  la  not  reatricted  to  the  narrow  field  of  wind  analyala. 
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IDENTIFICATION  OF  WORKERS  IN  BIOLOCICAIS  THROUGH 
SERUM  TITERS  BY  DISCRIMINANT  FUNCTION 
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SUMMARY.  Researchers  who  work  with  Infectious  biological  material  such 
as  Pasteurella  turarensls  generally  are  Immunised  lest  they  become  infected 
as  a  result  of  a  laboratory  mishap.  The  standard  measure  of  Immunity  la  the 
worker's  serological  antibody  titer  —  the  higher  titers  generally  thought  to 
confer  a  greater  degree  of  Immunity.  A  recent  experiment  compared  201  workers 
In  blologlcals  who  received  Immunological  procedures  designed  to  protect  them 
In  their  areas  of  specialisation  with  100  Army  draftees  who  received  the  stan- 
dard  Army  Immunisation  series.  Serological  titers  to  19  different  biological 
agents  were  assayed  for  each  of  the  persons  In  the  two  populations.  It  was 
Uio  purpose  of  this  paper  to  discover  if  the  parsons  could  be  correctly 
classified  Into  the  appropriate  group  according  to  their  serological  titers  only. 

A  preliminary  screening  for  agents  that  showed  group  differences  was  per¬ 
formed  by  using  the  t-tesc  on  group  means  for  each  titer  separately  and  by 
using  tha  chi-square  statistic  as  a  test  for  a  shift  in  frequency.  Candidate 
discriminators  were  those  agents  for  which  the  t-test  or  both  the  t-tast  and 
tha  chl-aquare  statistic  were  significant  at  the  .10  level.  Agenta  1,  2,  4, 

6,  7,  8,  15,  17,  19  survived  the  screening  procedure. 

A  modified  step-wise  discriminant  function  procedure  was  employed  to  create 
a  battery  of  discriminant  functions  that  competed  for  first  place  In  ability  to 
correctly  classify  persons  In  the  two  groups.  Because  the  sum  of  squares  attri¬ 
butable  to  the  discriminant  function  depends  upon  which  agents  wavs  Included, 
the  addition  of  another  agent  frequently  resulted  in  an  Improvement  In  the 
percent  classification  In  one  group  but  a  decrease  for  the  other. 
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Three  other  agents  5,  8,  end  13  (two  of  which  had  not  eurvlved  the  pre¬ 
liminary  screening)  woiu  observed  to  have  distributions  euch  that  ell  "control" 
group  members  (Army  rsftsas)  and  all  but  a  few  In  the  "treatment"  (biological 
workers)  group  had  negative  ticers.  Three  "decision  rules"  were  added  to  the 
classification  technique:  a  positive  titer  for  any  of  the  three  agents  5,  8, 
or  13  automatically  classified  the  person  as  "treatment";  a  negative  titer  for 
ell  three  called  up  the  discriminant  function.  A  small  but  distinct  improve¬ 
rs  :  the  percent  classification  was  noted  with  the  addition  of  the  decision 

.  ui 


a  cost  function  was  developed  aa  a  criterion  for  selecting  the  beet  of  the 
competing  discriminant  functions.  Costs  of  mlsclaaa if ylng  a  true  treatment 
person,  mlsclasslfylng  <»  true  control  person,  and  the  cost  of  assay  were  Included 
In  the  function.  Because  actual  costs  were  difficult  to  estimate.  It  was  valid 
to  recast  the  cost  function  In  terms  of  relative  costs  --  the  ratio  of  costs. 

The  battery  of  compe.lug  discriminants  was  evaluated  for  each  of  14  combinations 
of  values  allotted  to  the  three  cost  ratios  by  a  panel  of  experts. 

All  discriminants  In  a  battery  were  ranked  for  each  of  the  14  parameter 
combinations.  The  discriminant,  1-6-4-15,  plus  the  decision  agents  5,  8, 

13  was  found  to  ba  the  winner  for  99X  of  the  range  Included  in  the  cost  ratio 
parameters.  This  discriminant  was  able  to  classify  correctly  81%  for  the  control 
group  and  98. 5%  for  the  treatment  group.  Discriminant  1-6-4-2-15-17  was  also 
consistently  near  the  top  for  99%  cf  the  range  of  the  cost  ratios;  Its  percentages 
were  88X  and  97.5%  respectively.  That  it  was  not  a  winner  was  due  to  the 
emphasis  on  correct  classification  for  the  treatment  group. 

When  the  mlaclasslf lcatlon  coats  ware  equal,  the  ranking  of  the  discriminants 
was  sensitive  to  the  relative  cost  of  the  assay.  For  expensive  assays,  agent  1 
by  Itself  woe  beet;  for  Inexpensive  assays,  discriminant  1-6-4-2-15-17  plu9 
5,  8,  end  13  was  bast.  The  consistently  high-ranked  discriminant  mentioned 
above,  1-6-4-2-15-17  was  also  high-ranked  for  both  assay  costs. 

INTRODUCTION.  The  extent  to  which  a  serological  profile  might  "fingerprint" 
worker*  with  biological  materials  was  the  criterion  for  using  two  groups  of 
parsons  in  ths  Department  of  Army  for  sxtsnslve  serum  testing. 

Persons  known  to  bs  working  actively  with  blologlcals  were  selected,  100 
st  the  Army  Biological  Laboratories  st  Fort  Detrick;  and  101  from  workers  at 
Dugvsy  Proving  Ground,  a  testing  site.  A  control  group  of  ICO  enlisted  men 
receiving  the  standard  army  immunisation  series  but  known  not  Co  be  involved 
with  blologlcals  was  selected  at  Edgewood  Arsenal,  Maryland.  It  Is  the  purpose 
of  this  part  of  the  investigation  to  discover  If  a  combination  of  tltsrt  is 
more  affective  In  Identifying  workers  in  blologlcals  than  Individual  tltars  and 
to  what  extent  such  Identification  Is  predictable,  using  initially  the  data 
from  these  two  groupa. 
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Individual*  In  tha  Derr ick-Dugvay  "treatment"  group  were  selected  on  the 
baala  of  chair  active  participation  with  blologicils  .*nd  the  immunisation 
program*  provided  for  the  protection  of  the  workers  in  both  locations.  Serum 
titer*  to  19  microbiological  agent*  were  assessed  for  HI,  CF,  or  SN  antibodies 
for  each  iudlviw'ial;  an  extensive  questionnaire  was  requested  for  each  partici¬ 
pant  designed  to  p.ovida  information  that  might  account  for  titers  not  explain¬ 
able  by  ~it tmr  Immunisation  or  laboratory  contact. 

ggfjgy'l  \Y  SCB.KE.nNG .  If  tha  distributions  of  tltert  for  a  particular 
agant  worm  exacly  tha  aame  tor  the  two  population  groups,  t  re  would  be  no 
basis  for  usli'4  that  agent  to  classify  a  person  into  either  group.  A  perfect 
baala  would  b«  an  agar.'  for  which  there  was  alwavH  a  titer  in  the  treatment 
group  ;.nd  rav«r  a  titer  in  the  confol  group.  Another  perfect  basis  would 
exist  it  th;*  liter  distributions  for  the  two  groups  did  not  overlap.  The 
problem  becomes  one  of  finding  how  much  the  treatment  group  distribution  of 
titers  tor  an  a»*.,c  must  differ  from  that  for  the  control  group  to  be  an 
acceptable  basis  for  claaslf ic&c  on. 

A  two-9tage  procedure  was  devised  to  answer  the  question  of  which  agent9 
should  he  considered  as  effective  classifiers.  The  first  stage  has  been  called 
a  preliminary  screening  stage  and  consisted  of  computing  t-tests  on  the  mean 
positive  titers  for  the  two  groups  for  every  agent  and  a  chi-square  test  on  the 
frequency  of  specific  titers  for  every  agent.  The  results  of  this  preliminary 
screening  can  be  seen  in  Table  2.  These  two  tests  were  used  In  conjunction 
with  each  other  because  of  the  definitions  of  "positive"  and  "specific"  titer. 

A  positive  titer  was  any  titer  that  was  not  considered  negative.  When  the 
t-test  was  performed  on  the  means  ot  positive  titers,  peraouo  with  negative 
titers  were  not  Included  in  the  t-test  computation  for  that  agent.  However, 
they  were  Included  in  the  frequency  test  afforded  by  the  hi-square  statistic. 

Values  of  "specific"  titers  were  defined  arbitrarily  for  each  agent;  it 
was  Intended  that  they  should  represent  a  threshold  titer  not  likely  to  have 
occurred  because  of  assay  variability.  Values  of  both  are  shown  in  Table  2. 

The  probability  levels  for  these  two  statistical  tests  were  not  stringent; 
as  a  screening  device,  the  , 1C  level  was  used  as  opposed  to  the  .05  or  .01  levels. 
Those  agents  shewing  sufficient  distributional  differences  between  the  two  groups 
on  the  basis  of  both  tests  or  of  the  t-test  alone  were  Included  as  candidate 
discriminators.  Agents  1,  2,  <• ,  6,  7,  8,  15,  17,  19  survived  the  screening 
procedure,  A  parenthetical  note  is  added  here  to  note  chat  differences  between 
the  Duguay  and  Detrick  workers  were  also  examined  before  they  were  pooled  to 
form  the  "treatment"  group.  Several  dliferences  did  occur,  but  they  were 
generally  explainable  on  the  basis  of  know.,  subject  history.  There  was  no  valid 
ration  for  not  putting  these  two  groups  together  because  they  did  represent 
workers  in  microbiology. 
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The  second  stage  consisted  of  using  e  discriminant  function  --  a  combination 
of  Individual  agents  ~  as  a  basis  for  claaalf icatlnn.  Bofora  this  approach  is 
described.  It  la  worthwhile  examining  the  discriminating  ability  of  the  candi¬ 
date  agents  individually.  Table  3  lists  some  of  the  individual  agents  and 
indicates  the  percent  correct  classification  for  each  group. 

The  procedure  for  classification  was  simple.  The  mean  of  the  Agent  #1 
titer  (in  log  scale)  for  the  control  group  was  .9289  and  2.920  for  the  treatment 
group.  Halfway  between  the  two  means  is  the  mldmeen  value  1.925,  which  wee 
used  as  the  classifying  rule!  a  person  with  a  titer  less  then  the  midmean  was 
classified  as  control;  a  person  with  a  higher  titer  was  classified  aa  treatment. 
Since  each  person  was  known  to  be  in  one  group  cr  the  other,  the  percent  correc. 
classification  for  each  group  was  easy  to  compute  f  jr  one  agent  individually. 

Some  of  the  agents  with  extremely  high  percentagcc  for  one  group  but  low  for 
the  other  deserve  a  word  of  comment.  Agent  6,  for  example,  gave  98  and  10  percent 
correct  for  control  and  treatment  groups  respectively.  Actually,  only  22X  of 
the  subjects  in  the  control  group  and  10X  of  the  subjects  In  the  treatment  group 
had  positive  titers.  Thus  96X  of  the  control  group  was  correctly  classified; 
those  in  the  treatment  group  with  titers  less  than  the  midmean  were  also 
classified  as  control  and  therefore  Incorrectly.  This  agent  could  not  be 
considered  very  effective  according  to  the  midmean  rule;  nevertheless,  it  was 
retained  for  possible  use  either  in  the  discriminant  function  or  in  the 
decision  rule  approach. 

DISCRIMINANT  FUNCTION  APPROACH.  A  linear  discriminant  function  is  a 
combination  of  predictors  (agents),  each  with  a  weight  con  expending  tc  its 
relative  effectiveness  in  discrimination: 

F  -  B, X,  +  B.X,  ♦  ...  +  B.X. 

11  2  2  k  k 

The  X  ere  the  titer  values  In  logarithms  for  the  various  agents,  1— k,  used 
in  the  discriminant;  the  weights,  B^,  are  estimated  from  the  data.  As  applied, 
e  person's  titers  (in  log  scale)  are  substituted  as  the  X  values;  if  the 
resulting  F  value  is  closer  to  that  for  the  control  group  than  it  is  for  the 
treatment  group,  the  subject  is  classified  as  belonging  to  control.  Theory 
guarantees  that  the  methodology  for  estimating  the  weights  results  in  the  bast 
possible  linear  discriminant  using  these  predictors. 

The  individual,  one-agent-only  approach  described  earlier  with  the  results 
shown  in  Table  3  cannot  apply  to  a  combination  of  agents.  The  presence  of  a 
second  and  more  discriminators  in  conjunction  with  the  first  will  have  definite 
effects  on  the  behavior  of  the  discriminant  function.  The  method  of  estimating 
the  weights  for  the  linear  discriminant  above  recognises  these  covariable  effects 
and  optimises  their  Joint  function. 


APPLICATION  OF  THE  DISCRIMINANT  FUNCTION.  Perhaps  the  aaalast  way  to  look 
at  the  reaulta  of  tKa  iiecriminant  Functions  as  constituted  by  a  variety  of 
predictor  combinations  la  tc  look  at  the  percent  correct  classification  for  each 
group.  Table  4  identifies  the  various  discriminants  In  terms  cf  the  agents 
ueed  aa  predictors  and  shove  the  percent  correct  claaslf lcetion  for  each;  they 
may  be  regarded  as  competing  for  optimum  performance.  The  most  obvious  result 
was  the  clear  superiority  of  all  discriminants  with  agent  #1  over  thoae  without 
#1.  This  finding  was  true  regardless  of  the  number  of  other  agents  or  which 
ones.  Thus,  the  next  step  was  to  select  the  beat  discriminant,  having  discarded 
all  discriminants  without  agent  #1. 

The  question  of  which  discriminant  to  choose  was  complicated  by  the  fact 
that  some  discriminants  did  better  for  one  group,  and  others  did  better  for  the 
other  group;  none  was  clearly  best  for  both  groups.  A  second  complication  arose 
by  virtue  of  the  Inherent  characteristics  of  discriminant  function  estimation. 

For  example,  as  can  be  seen  In  Table  4,  the  addition  of  agent  #4  to  discriminant 
function  1-6  resulted  in  going  from  percent  correct  classification  94.0  and 
88.6  for  the  control  and  treatment  groups  respectively  to  90.0  and  91.0.  Thus 
the  addition  of  another  variable  to  a  discriminant  function  was  not  necessarily 
an  improvement  in  both  categories.  Further  study  of  Table  4  does  not  allow  an 
easy  Identification  of  the  winning  diacr iminant.  A  discussion  of  criterion 
for  choosing  is  deferred  until  the  approach  involving  decision  rules  has  been 
added  to  Che  discriminant  function  approach. 

DECISION  RULES  AS  AN  ADJUNCT  DISCRIMINATION  DEVICE.  An  additional  device, 
denoted  here  as  "d»clflion  rules,"  utilised  two  agVnts'not  already  incorporated 
In  any  of  the  discriminant  function*.  This  approach  is  analogous  to  the  fre¬ 
quency  basis  for  tne  preliminary  screening;  the  discriminant  function  corresponds 
to  the  comparison  of  means.  Two  of  the  agents,  45  and  13,  were  not  deemed  to 
be  potentially  useful  for  the  disci  iu*i. ......  function  approach  because  of  Che 

generally  low  Incidence  of  positive  titers  in  either  group.  Howevpr,  these  two 
agents  with  agent  #8  shared  one  characteristic  in  consnon  —  all  titers  for  all 
control  subjects  were  negative  (with  the  exception  of  2  persons  for  agent  48). 

To  take  adventage  of  the  Information  afforded  by  the  distributional  form,  the 
following  decision  rules  were  set  up  as  an  adjunct  to  the  discriminant  function 
approach  as  an  enlarged  basis  for  classifying  subjects  into  the  control  and 
treatment  groups: 

1)  If  a  person  has  a  positive  titer  to  agent  45,  classify  him  as  treatment 
and  remove  the  subject  from  further  classifying  rules.  If  the  subject  has  a 
negative  titer,  do  not  classify  but  go  on  to  agent  48. 

2)  It  the  subject  has  a  positive  titer  to  agent  48,  classify  as  treatment 
and  remove  subject  from  furthet  classifying  rules.  If  he  has  a  negative 
titer,  do  not  classify  but  go  on  to  ag»nt  413. 
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3)  If  subject  has  a  positive  titer  to  agenw  #13,  classify  as  treatment 
and  remove  subject  from  further  classifying  rules.  If  he  has  a  negative  titer, 
do  not  classify  but  apply  the  discriminant  function. 

This  decision-rule  approach  is  essentially  probabilistic  —  one  that  might 
be  developed  much  further  by  means  of  joint  and  conditional  probability  state¬ 
ments.  This  opportunity  has  not  been  exploited  beyond  the  rules  described  above. 

the  effectiveness  of  the  decision  rule  approach  compared  with  the  discriminant 
function  only  can  be  seen  in  Table  S.  The  results  were  consistent.  The  control 
group  percentages  uniformly  decreased  by  2X,  obviously  because  of  the  two  control 
subjects  with  positive  titers  to  agent  #8.  However,  the  percent  correct  classifi¬ 
cation  for  the  treatment  group  increased  without  exception.  If  there  is  greater 
importance  attached  to  correct  classification  for  the  treatment  group,  then  the 
addition  of  the  decision  rules  generally  was  a  benefit.  The  problem  of  selecting 
a  best  combination  among  the  discriainant3  with  or  without  the  decision  rules 
remains  to  be  answered.  Again,  there  has  been  no  obviously  established  winner 
after  the  addition  of  the  decision  rules.  A  criterion  for  selection  is  clearly 
needed. 

RELATIVE  COST  AS  A  BASIS  FOR  A  CRITERION  OF  EVALUATION. 

A.  Development  of  the  Cost  Function 

A  simple  rule  or  criterion  would  be  to  consider  the  percent  correct  classi¬ 
fication  for  each  group  to  be  of  equal  importance.  If  no  other  rule  were  added, 
then  the  winner  along  the  first  six  functions  listed  in  Table  4  would  be  function 
1-8,  whoso  average  percent  correct  classification  for  the  two  groups  was  9c. 0, 
higher  than  any  other  average  in  the  first  six.  However,  the  average  percent 
correct  classification  for  function  1-6-4-2  was  92.5,  but  it  required  the 
addition  of  two  more  agents.  The  question  arises  whether  the  Increase  in  the 
average  percentage  was  worth  the  cost  of  the  additional  assays,  A  general  model 
of  cost  as  a  criterion  is  ueveloped  in  this  section,  with  application  to  these 
questions:  (1)  was  the  addition  of  the  decision  rules  worthwhile  and  (2)  what 
is  the  most  efficient  discriminant  when  performance  is  assessed  against  cost? 

In  this  development  of  a  cost  function,  it  will  be  convenient  to  define 
certain  costs  without  stating  or  defining  their  dollar  value.  Wc  begin  by 
defining  error  rates  as  another  way  of  stating  successful  classification: 

Let  =  proportion  correct  classification  in  the  contro1  group  and 
P  ■  proportion  C'rre.t  classification  in  the  treatment  group. 
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Then  the  error  rates  or  rates  of  mlsclaaalf lcatlon  are  defined  as 

qc  •  1  -  pc»  proportion  of  Incorrect  classification  for  control  and 

qt  ■  1  -  p  •  proportion  of  incorrect  classification  for  treatment. 

Let  *  cost  of  mlaclasslfylng  a  control  subject  and 

Q(  •  cost  of  alsclassifying  a  treataent  subject. 

In  addition  to  converting  error  rates  into  cost  of  alsclasslf icatlon  as  shown 
above,  we  proceed  to  include  the  cost  of  assay  in  the  cost  function. 

Let  B  *  cost  of  one  bacterial  assay  and 

V  -  cost  of  one  viral  (or  rickettsial)  assay  by  hemagglutination  inhi¬ 
bition. 

Let  b  «  number  of  bacterial  agents  used  in  discriminator  and 

v  -  number  of  viral  (or  rickettsial)  agents  used  in  discriminator.  There¬ 
fore, 


bB  »  cost  of  b  bacterial  assays  and 

W  -  cost  of  v  viral  assays. 

Thus  Cost  •  q  0  +  q,Q.  +  vV  +  bB  . 
c  c  c  c 

This  statement  adds  both  costs  of  alsclasslf lcatlon  to  the  cost  of  assay.  The 
reduction  of  any  of  the  quantities  in  the  cost  equation  results  In  reducing  the 
overall  cost.  Specifically,  if  the  quantities  q  and  q  are  reduced,  the 
resulting  improvement  in  correct  classification  San  be  translated  into 
reduced  costs. 

But  It  may  be  difficult  to  specify  the  actual  dollar  values  for  the  quantities 
Q  ,  Qt,  V,  and  B.  Therefore,  It  may  be  mors  convenient  end  still  relevant  to 
define  relative  costs  as  follows: 

Let  Qt/Qc  •  k,  0  <  k  <  ■>,  be  the  cost  of  if isclasslfylng  a  treatment  subject 

relative  to  the  cost  of  misclassifying  e  control 
subject.  Likewise, 

Let  V/B  -  c,  0  <  c  <  be  the  ccat  of  a  viral  (or  rickettsial)  assay  relative 

to  the  cost  of  a  bacterial  assay.  And  finally. 

Let  Qc/B  •  r,  0  <  r  <  »,  be  the  cost  of  mlsclaselfylng  a  control  subject 

relative  to  the  cost  of  a  bacterial  assay.  Upon 
substitution  of  these  terms  into  the  coet  equation 
above,  we  have 
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Relative  Coat  “  qcQc  +  kq£Qc  ♦  cvB  +  bB 

■  rqcB  +  rkqtB  +  cvB  +  bB  and 
(Relative  Cost)/B  *  r(qc  +  kq£)  +  cv  +  b. 

This  expression  Cor  (Relative  Coat)/B,  a  dimensionless  quantity,  now  depanda 
only  on  the  two  error  rates,  q  and  q  ,  and  the  number  of  assays.  It  continues 
to  be  a  criterion  for  excellence;  the  lower  the  cost,  the  better  the  discrim¬ 
inator.  The  result  of  dividing  Relative  Cost  by  the  factor  B  has  no  effect  on 
the  criterion  because  B  has  been  defined  as  a  constant.  Evaluation  of  the 
various  discriminants  does  depend  upon  the  values  selected  for  the  constants, 
r,  k,  and  c.  Of  these,  It  was  easy  to  give  a  value  to  only  one  ratio,  c.  It 
was  deemed  that  the  cost  of  an  HI  test  was  essentially  equivalent  to  the  costs 
of  ocher  methods  of  assay  (excluding  serum  neutralisation);  therefore  the 
value  of  c  was  set  as  unity.  Rather  than  use  a  single  value  for  r,  two  values 
were  suggested  by  a  panel  to  represent  a  reasonable  range;  these  were  50 
and  250.  Because  the  ratio  of  the  cost  of  mlsclasslf icstion  was  both  difficult 
to  asalgn  a  value  to  and  because  the  quantity,  (Relative  Coat)/B,  was  very 
sensitive  to  It  for  low  values,  three  values  were  assigned  Initially  to  k  — 
k  ■  1,  30,  1000—  to  rover  a  wide  range. 

B.  Evaluation  Results  for  the  Discriminant  Functions  Only 

To  evaluate  every  one  of  the  discriminants  listed  In  Table  4  would  Include 
many  discriminants  with  obviously  poor  performance,  namely  those  without 
agent  01.  Therefore,  only  those  discriminants  that  Included  agent  #1  ware 
considered  for  the  cost  function  analysis.  Every  discriminant  that  included 
Agent  #1  but  omitted  the  decision  rules  was  evaluated  for  all  combinations  of 
the  ratio  constants  c,  k,  and  r.  Results  of  these  evaluations  are  given  in 
Table  6,  There  It  can  be  6een  that  for  both  r  values  and  the  two  higher  values 
of  k,  the  discriminant  consisting  of  agents  1-6-15  was  consistently  top  ranked. 
Running  second  and  third  were  discriminants  1-6-4-15  and  1-6-4-15-17.  The 
performance  of  these  first  three  discriminants  as  ranked  In  cost  and  measured 
by  percentage  correct  classifications  were  80-96,  83-95.5,  and  80-95.5  respec¬ 
tively  for  the  control  and  treatment  groups. 

There  was  no  effect  In  changing  the  value  of  r  from  50  to  250  for  the  two 
higher  values  of  k  on  the  ranking  of  the  first  five  discriminant  functions; 


1-6-15 
1-6-4-15 
1-6-4-15-1 7 
1-6-4-2 
1-15 

which  had  the  same  order  In  the  four  rankings  Involved. 
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However,  char*  «u  a  draatlc  ravaraal  of  the  ranking*  whan  ch*  value  of 
unity  was  assigned  to  k.  The  flrat  three  function*  referred  to  above  did 
not  appear  at  all  In  the  flrat  20  ranks  for  either  r  ■  50  or  r  ■  250. 

Because  of  this  reversal,  further  investigation  of  the  effect  of  k  on  the 
ranking  of  the  discriminants  was  deferred  until  the  addition  of  the  declalon 
rules  has  been  evaluated  by  the  coat  function. 

C.  Evaluation  Results  for  Discriminant  Functions  Plus  Decision  Rules 

The  sane  coat  function  developed  above  was  applied  to  the  raaulta  of  the 
dlscrlalnant  function  plus  decision  rules  for  all  coabinatlona  of  c,  k,  and 
r.  These  are  ranked  by  Increasing  coat  In  Table  7.  Two  results  are  Immediately 
apparent,  there  was  an  laprovaaenc  In  daseif ication  according  to  the  cost 
function  criterion  with  the  addition  of  the  decision  rules  for  the  values  of 
k  ■  30  and  k  ■  1,000  by  virtue  of  the  decreased  coats.  This  was  not  so  for 
values  of  k  ■  1  and  both  values  of  r,  where  only  the  dlscrlalnant  function  had 
generally  lower  costs  than  dlscrlalnant  plus  decision  rules  and  therefor*  was 
a  more  effective  dlscrlninator.  This  result  stemmed  from  comparing  the  entries 
In  Table  6  vs  Table  7  rank  by  rank  for  each  of  the  six  categories. 

The  second  result  was  the  fact  that  the  top  rankings  with  or  without  the 
decision  rules  tended  strongly  to  be  the  sane  for  values  of  k  ■  30  and  k  “  1,000. 
For  example  In  the  four  cases  k  -  30,  k  ■  1,000  for  both  values  of  r,  the  top 
three  rankings  were  composed  of  the  sen*  discriminant  functions.  For  the  top- 
ranked  six  discriminant  function  plus  decision  rules  In  Table  7,  five  of  these 
Six  discriminant  functions  were  also  top-ranked  In  the  first  six  places  for 
dlscrlalnant  function  without  the  decision  rulest  this  result  held  for  the 
same  four  casea  of  -  •  30,  k  -  1000  for  both  values  of  r.  Not*  that  costs 
are  not  to  be  compared  across  categories  of  k  and  r;  they  are  relevant  only 
within  a  given  category. 

The  results  Including  th*  decision  rules  also  encountered  the  previous 
expedience  of  changing  drastically  when  k  ■  1.  Therefore  It  la  worthwhile 
investigating  the  behavior  of  the  cost  function  as  It  Is  affected  by  changes  In 
th*  parameter  k,  especially  In  the  region  between  k  •  30  and  k  -  1,000  where 
the  large  reversals  occurred  as  notsd  above.  Arbitrarily,  th*  value  of  r  was 
kept  fixed  at  r  -  250  while  Intermediate  values  of  k  *  20,  10,  5,  and  2  were 
added  to  the  original  set  of  k  ■  1,000,  30,  and  1.  The  ranking  of  the  dls¬ 
crlalnant  functions  plua  decision  rules  wss  obtslnad  for  each  value  of  k. 

These  are  shown  In  Tabla  8  where  It  Is  obvious  that  tha  ranking  at  k  ■  20 
remained  Identical  to  that  for  k  -  30  and  k  -  1000.  A  small  shift  In  ths  ranks 
occurred  at  the  value  k  •  10;  nuch  larger  disruptions  occurred  for  value*  of 
k  lower  than  10.  As  a  method  of  looking  quantitatively  at  the  degree  of  change 
In  the  ranks  with  the  change  In  k,  it  wsa  convenient  to  compute  Spearman'n 
Coefficient  of  Rank  Correlation  with  k  ■  1,000  aa  the  base  line  successively 
with  rankings  for  all  other  values  of  k.  A  plot  of  Spearmen's  Rho  versus  k  Is 
ehown  In  Figure  1.  It  may  be  concluded  for  r  ■  250  chat  the  ranking*  of  the 
discriminants  are  essentially  unchanged  for  values  of  k  larger  than  10.  Thus 
the  choice  of  a  winning  discriminant  plus  decision  rule  for  thoee  considered 
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hire  can  ba  baaad  on  a  alapla  Judgment  aa  ro  whether  k  ahould  ba  considered 
smaller  than  10.  For  tha  eaaa  of  k  ■  10  or  more,  tha  conalatant  number  ona 
oarforaar  la  baaad  on  tha  dlacrlalnant  composed  of  agents  1-6-4-15  and  tha 
decision  rulaa  baaad  on  aganta  St  8,  and  13. 

Tha  temptation  to  raach  down  to  dlacrlalnant  function  1-15,  which  la 
rankad  In  fourth  place  for  the  hlghar  valuaa  of  k  and  uaa  It  on  tha  baala  of 
tha  laplled  aconoay  of  only  two  aaaaya  (plua  tha  thraa  for  tha  daclalon  rula) 
ahould  ba  raalatad.  Slnca  tha  coat  function  haa  alraady  Included  tha  coat 
of  tha  aaaaya,  a  aelactlon  of  othar  than  tha  winning  combination  would  ba 
baaad  on  falaa  premises. 
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TABU  2. 

A*ent  # 

PRELIMINARY  SCREENING  BY  GEOMETRIC  KEANS  AMD  FREQUENCY 

Daflnltlon  Daflnltlon  Control  Treatment 

of  of  G-mean  G-Hean  t-taat 

Poaltlva  S pacific  Poaltlva  Poaltlva  on 

TUnr*1  Tltar _ Tltor" _ Titer’ _ C-naaot _ 

x2-teat 

on 

Pranuancv 

Tentative 

itii 

1 

10 

80 

24.3 

201. 

19.0** 

188.** 

X 

2 

25 

200 

39.9 

52.6 

2.9** 

2.6(1) 

X 

3 

2 

2 

7.6 

9.3 

.8 

19.4** 

4 

5 

80 

128. 

251. 

3.1** 

7.4** 

X 

5 

5 

10 

0. 

13.5 

29.7** 

6 

5 

10 

5.0 

8.9 

4.3** 

4.7* 

X 

7 

5 

10 

8.6 

5.7 

1.7(1) 

2.3(2) 

X 

8 

2 

2 

2.8 

6.0 

2.1* 

46.5** 

X 

9 

20 

40 

0. 

30.3 

.2 

10 

20 

40 

44.1 

41.6 

.6 

2.7(1) 

11 

20 

40 

0. 

26.4 

.3 

12 

20 

40 

20.0 

24.8 

1.3(2) 

.3 

13 

20 

40 

0. 

32.6 

13.8** 

14 

15 

20 

40 

22.4 

27.8 

2.6** 

2.3(2) 

X 

16 

20 

40 

0. 

23.3 

.7 

17 

20 

40 

20.0 

71.3 

2.4* 

1.1 

X 

18 

20 

40 

0. 

0. 

19 

20 

40 

20.0 

29.0 

1.8(1) 

1.1 

X 

**  Statistically  significant  at  .01  laval 
*  Statistically  significant  ac  .05  laval 

(1)  Statistically  significant  at  ,10  laval 

(2)  Statlatlcally  algnlflcant  at  .20  laval 

G-naan:  Geometric  mean  of  poaltlva  tltara. 

2 

Frequency:  x  on  fraquancy  of  specific  tltara. 
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TAU  #3.  PUtCW IT  COKRXCT  CLASSIFICATION  BY  SOME  AGENTS  INDIVIDUALLY 


Afaat  # 

Par cant  eorraet 

classification 

control 

Percent  correct 

classification 

traatnsnt 

1 

94.0 

88.1 

A 

98.0 

10.0 

4 

52.0 

63.7 

2 

49.0 

59.7 

IS 

43.0 

61.2 

17 

99.0 

3.0 

8 

98.0 

39.3 
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1-6-4-2-15-17-8-19-7  92.0  94.5 


OQWAUKW  or  OOKMCT  CUSSIFICATIOHt 
OXNMNZIUIIT  TUICTIOH  OHLT 


D8CX810M  HULKS  V8 


A|iat  #'• 

Dlocrlnioant 

Function  Only 

01 ter ini  runt 
Docialoa 

Function  + 

Hu  loo 

Control 

Troataont 

Control 

Trootanat 

1-4 

85.0 

94.5 

81.0 

97.0 

l-l 

89.0 

94.0 

87.0 

97.0 

1-15 

81.C 

95.0 

79.0 

98.0 

1-4-2 

90.0 

94.0 

88.0 

97.0 

1-4- 15 

80.0 

96.0 

78.0 

98.5 

1-4-4-2 

90.0 

95.0 

88.0 

97.5 

1-4-4- IS 

85.0 

95.5 

81.0 

98.5 

1-4-2-15 

89.0 

94.0 

87.0 

97.5 

1-4-2-17 

90.0 

94.5 

88.0 

97.0 

1-4-2-17 

90.0 

94.5 

88.0 

97.0 

1.6-4-2-15 

90.0 

94.0 

88*.  0 

97.5 

1.4-4-2-17 

91.0 

93.5 

89.0 

96.5 

1-4-4-15-17 

80.0 

95.5 

78.0 

98.5 

1-4-2-15-17 

90.0 

94.0 

88.0 

97.5 

1-4-2-15-17 

90.0 

92.5 

88.0 

96.5 

1.6-4-2-15-17 

92.0 

95.5 

90.0 

97.5 

1-6-4-2-15-17-8 

94.0 

94.5 

92.0 

97.0 

1-4-4-2-15-17-8-1? 

92.0 

94.5 

90.0 

97.0 

1.6-4-2-15-17-8-19-7 

92.0 

94.5 

90.0 

97.0 
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TAILS  I.  CHANGE  IN  RANKINGS  OP  DISCRIMINANT  r UNCTION 8  PLUS  DECISION  RULES 
BY  COST  POX  CHANGES  IN  k 

c  •  li  r  •  ISO 


Valuta  of  k 


Agant  #'a 

1000 

30 

20 

10 

3 

2 

1 

1-6.4-15 

1 

1 

1 

1 

7 

IS 

15 

1.6-13 

2 

2 

2 

3 

15 

18 

18 

1-6.4-13-17 

3 

3 

3 

7 

17 

19 

19 

1-13 

4 

4 

4 

10 

18 

17 

17 

1.6-4-2-13-17 

5 

5 

5 

2 

1 

2 

2 

1-6*4. 2 

6 

6 

6 

4 

3 

3 

3 

1-6.4-2.13 

7.5 

7.5 

7.5 

3.3 

4.3 

4.3 

7.5 

1-6-2-13-17 

7.5 

7,5 

7.5 

3.5 

4.3 

4.3 

7.5 

1-6-2-13 

9 

9 

9 

8 

6 

11 

13 

1-6-4-2-15-17-8 

10 

10 

10 

9 

2 

1 

1 

1-6-2 

11.5 

11.3 

11.3 

11.5 

8.5 

6.5 

4.3 

1-6-4-2-15-17-8-19 

11.3 

11.3 

11.5 

11.5 

8.3 

6.3 

4.5 

1-6-2-17 

14 

14 

14 

14 

11 

9 

10 

1-4-2-17 

14 

14 

14 

14 

11 

9 

10 

1-6-4-2.15-17-8-19-7 

14 

14 

14 

14 

11 

9 

10 

1-2 

16 

16 

16 

16 

13 

12 

12 

1-4 

17 

17 

17 

19 

19 

16 

16 

1-6-4-2-17 

18 

18 

18 

17 

14 

13 

6 

1-4-2-15-17 

19 

19 

19 

18 

16 

14 

14 

Valuaa  of  Sptarmaa'a  Rho 

1.00 

1.00 

.928 

.298 

-.049 

-.168 
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SN  ED  EC  OR  AWARDED  THE  1970  SAMUEL  S.  WILKS  MEMORIAL  MEDAL 


Professor  George  Snedacor,  now  retired  from  Iowa  Stata  Univaraity, 
has  bean  awarded  the  Samuel  S.  Wilks  Manorial  Medal  for  1970  The 
announcanent  of  Profeaaor  Snadecor'a  select  ion  for  the  1970  Wllka  Award 
was  one  of  the  highlights  of  the  Sixteenth  Annual  Conference  on  the 
Design  of  Experiments  In  Army  Research,  Davalopaant  and  Tasting,  which 
was  held  at  the  U.  S.  Army  Logistics  Management  Center,  Fort  Lee,  Virginia, 
21-23  October  1970.  Profeaaor  Snedacor  has  long  baan  recognised  aa  a 
pioneer,  an  International  authority  and  one  of  the  outstanding  applied 
statisticians  in  both  the  U.  S.  A.  and  countries  abroad  aa  well,  having 
made  many  fundamental  contributions  to  statistical  methodology  and 
analysis  of  statistical  experiments.  The  citation  for  Professor  Snedacor 
raada  aa  follows: 

"To  George  W.  Snedecor  for  his  pioneer  contributions 
In  the  develops**!^  and  use  of  statistical  methods.  Including 
applications  of  experimental  design  to  research  investigations, 
and  for  Introducing  several  generations  of  statisticians 
and  reaaarch  workers  to  the  subject  of  statistics  through 
teaching  and  the  six  editions  of  his  world-renowned  book, 

Statistical  Methods." 


Previous  recipients  of  the  Samuel  S.  Wilks  Memorial  Medal  Include 
John  W.  Tukey  of  Princeton  University  (1965),  Major  General  Leslie  E.  Simon 
(1966),  William  G.  Cochran  of  Harvard  University  (1967),  Jarsy  Neyman  of 
cha  University  of  California  (1968),  and  Jack  Youden  (1969)  retired  from 
the  National  Bureau  of  Standards. 

The  Samuel  S.  Wilks  Memorial  Medal  Award,  initiated  In  1964  by  the  U.  S. 
Army  and  the  American  Statistical  Association  jointly.  Is  administered  by  the 
American  Statistical  Association,  a  non-profit,  educational  and  scientific 
society  founded  In  1839.  The  Wilks  Award  is  given  each  year  to  a  statistician 
and  Is  based  primarily  on  his  contributions  to  the  advancement  of  scientific 
or  technical  knowledge  In  Army  statistics,  Ingenious  application  of  such 
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knowledge,  or  successful  activity  in  ths  fostering  of  cooparativa  aciantific 
matters  which  coincidentally  benefit  the  Any,  tha  Dapartaant  of  Defenee, 
tha  U.  S.  Government,  and  our  country  generally. 

The  Award  consists  of  a  medal,  with  a  profile  of  Professor  Wilks  and 
tha  naaa  of  tha  Award  on  one  side,  the  seal  of  tha  American  Statistical 
Association  and  name  of  the  recipient  on  the  reverse,  and  a  citation  and 
honorarium  related  to  the  magnitude  of  the  Award  funds.  The  annual  Army 
Design  of  Sxpariments  Conferences,  at  which  tho  Award  is  given  each  year, 
are  sponsored  by  tha  Army  Mathematics  Steering  Comittee  on  behalf  of  toe 
Office  of  the  Chief  of  Research  and  Development,  Department  of  the  Army. 

The  funds  for  the  S.  S.  Wilke  Memorial  Award  were  donated  by  Philip  G. 
Rust,  retired  industrialist,  Thomasvllle,  Georgia. 

With  the  approval  of  President  T.  A.  Bancroft  of  the  American  Statistical 
Association,  the  Wilks  Memorial  Medal  Committee  for  1970  coneieted  of  the 
following! 


Profeaeor  Robert  E.  Bethhofer 
Professor  William  G.  Ccchian 
Dr.  Francis  C,  Dressel 

Dr.  Churchill  Eisenhart 

Profeaeor  Oscar  Xempthorne 

Dr.  Alexander  M.  Mood 

Major  Ceneral  Leslie  E.  Simon 

Dr.  John  W.  Tukey 

Dr.  Frank  E.  Grubbs,  Chairman 


-  Cornell  University 

-  Harvard  University 

-  Duke  University  and  the  Army  Research 
Off ice- Durham 

-  National  Bureau  of  Standards 

-  Iowa  State  University 

-  University  of  California 

-  Retired 

-  Princeton  University 

-  U.S.  Army  Aberdeen  Research  & 
Development  Center 

Aberdeen  Proving  Ground,  Maryland 


George  W.  Snedecor  is  an  international  authority  who  was  a  plonaer  In 
his  field  of  statistics  and  related  sciences. 


Born  October  2,  1881  in  Memphis,  Tennessee,  he  received  hie  education 
at  the  Alabama  Polytechnic  Institute,  the  University  of  Alabama  (B.S.,  1905, 
mathematics  and  physics)  and  the  University  of  Michigan  (A.M.,  1913,  physics). 
He  joined  the  Iowa  State  University  faculty  as  an  assistant  profaasor  of 
mathematics  in  1913,  and  remained  on  campus  for  45  years. 

Shortly  after  hie  arrival  at  Iowa  State,  Professor  Snedecor  took  the 
lead  in  helping  people  with  problems  in  statistical  applications.  In  1924 
he  partlclapted  in  weekly  sessions  conducted  by  Henry  A.  Wallace,  then  editor 
of  Wallace's  Parmer.  At  these  sessions  rapid  machine  calculation  was 
demonstrated  as  a  method  of  solving  basic  statistical  needs. 
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As  a  result  of  these  get-togethers  a  bulletin,  "Correlation  and  Machine 
Calculation,"  was  written  In  1925  by  Snedecor  and  Wallace.  It  promptly 
attained  worldwide  distribution  as  a  pioneer  publication  In  statistics,  and 
remained  in  use  for  many  years. 

Because  of  his  lntereat  and  enthusiasm  In  the  field,  Professor  Snedecor 
continued  statistical  experimentation.  In  192?  he  and  A,  E.  Brandt  were  put 
in  charge  of  a  newly  created  Mathematics  Statistical  Service  at  Iowa  State 
University.  This  Service  was  established  to  supply  the  demand  for  professional 
help  in  statistics. 

Professor  Snedecor  had  ar.  appreciation  for  the  Importance  of  the  work 
of  the  eminent  British  statistician  Sir  Ronald  Fisher,  who  was  to  become 
known  as  the  greatest  figure  In  the  history  of  statistics.  As  a  result. 

Sir  Ronald  was  brought  to  Iowa  State  University  as  a  visiting  lecturer 
during  the  summers  of  1931  and  1936,  Iowa  State  was  the  first  l).  S. 
institute  to  award  Sir  Ronald  an  honorary  degree. 

In  1933  Iowa  State's  Statistical  Laboratory  was  established  as  a  research 
Institute  under  the  president,  with  George  W.  Snedecor  as  Its  first  director. 

It  was  the  firsr  statistical  center  of  its  kind  In  the  United  States,  and 
its  organization  provided  the  Impetus  for  other  universities  to  establish 
similar  research  and  service  institutes  In  statistics. 

As  an  administrator,  Professor  Snedecor  collected  c  staff  which  earned 
a  reputation  for  excellence.  His  diligence  in  developing  cooperative  agree¬ 
ments  between  the  Statistical  Laboratory  and  the  U.  S.  government  provided 
funds  for  the  expansion  of  Laboratory  personnel  staff  and  its  projects. 

As  a  teacher.  Professor  Snedecor  inspired  his  students  to  achieve  their 
highest  goals,  and  encouraged  them  to  establish  even  higher  goals.  The  work 
and  contributions  of  his  studenta  alone  have  given  him  a  reputation  as  one 
of  the  foremost  teachers  of  statistics  during  the  development  of  the  field. 

Professor  Snedecor  served  as  director  of  the  Statistical  Laboratory  until 
college  regulations  forced  him  to  relinquish  administrative  responsibilities 
In  1947  at  the  age  of  65.  During  that  same  year  a  separate  Department  of 
Statistics  was  established.  Snedecor  continued  as  a  professor  of  statistics 
on  a  part-time  basis  until  his  retirement  In  1958,  and  still  Is  affiliated 
with  the  university  as  professor  emeritus. 

Professor  Snedecor  Is  recognized  throughout  the  world  for  his  pioneer 
work  in  the  systematic  development  and  wide-spread  use  of  statistical  methods. 
Kia  book,  Statistical  Methods,  first  published  in  1937,  Is  now  in  Its  sixth 
edition  and  has  sold  more  than  100,000  copies.  It  has  been  translated  Into 
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Spanish,  Hindi,  Japanaas  and  Rumanian.  A  Franch  cranalatlon  ia  now  in 
progrsss  and  an  Indian  raprlnt  haa  baan  publlahad  in  Engiiah,  Snadacor  also 
ia  cha  author  of  thraa  othar  books  and  some  SO  papara  on  statistics. 

In  1948  Profassor  Snadacor  was  named  president  of  the  American  Statistical 
Association.  His  national  and  international  reputation  ia  indicated  by  his 
election  aa  a  Fallow  of  the  American  Statistical  Association,  the  Institute 
of  Mathematical  Statistic*,  and  the  American  Association  for  the  Advancement 
of  Science.  He  wea  elected  an  Honorary  Fellow  of  the  British  Royal  Statistical 
Society,  an  honor  given  to  only  a  very  few  Americana. 

He  holds  membership  in  the  International  Statistical  Institute,  the 
Biometrics  Society,  Iowa  Academy  of  Science,  Phi  Beta  Kappa,  Sigma  XI,  Gaaaa 
Alpha,  Pi  Hu  Epsilon  and  Kappa  Sigma. 

Ceorge  W.  Snedecor  received  the  Iowa  State  University  Faculty  Citation 
in  1955,  and  honorary  Doctorates  of  Science  from  North  Carolina  State 
University  in  1956  and  Iowa  State  in  1958.  On  May  18,  1970  the  building 
which  houses  Iowa  State's  statistical  center  was  formally  renamed  Snedecor 
Hall  in  his  honor. 

GEORGE  W.  SNEDECOR i  A  CHRONOLOGY 


October  20,  1881  -  born  in  Memphis,  Tennessee 

1899-1901  -  attended  Alabama  Polytechnic  Institute 
1901-1905  -  attended  University  of  Alabama 

1905  -  received  B.S.,  University  of  Alabama  (mathematics  and  physics) 
1905-1907  -  instructor,  Selma  Military  Academy 

1907-1910  -  professor,  mathematics,  Austin  College,  Sherman,  Texas 
1908  -  married  Gertrude  Crosier 

1910-1913  -  graduate  assistant,  physics.  University  of  Michigan 
1913  -  received  A.M.,  University  of  Michigan  (physics) 

1913- 1914  -  assistant  professor,  mathematics,  Iowa  State  University 

1914- 1930  -  associate  professor,  mathematics,  Iowa  State  University 
1915  -  first  courses  in  statistics  offered  at  Iowa  State  University 

1924  -  attended  a  tan-week  course  conducted  by  Henry  A.  Wallace  on  rapid 

machine  calculation  of  correlation  coefficients,  partial  correlation 
and  the  calculation  of  regression  lines 

1925  -  co-author,  with  Wallace,  of  "Correlation  and  Machine  Calculation," 

a  bulletin  which  attained  worldwide  distribution 
1927  -  In  charge,  with  A.  E,  Brandt,  of  a  newly  created  Mathematics 
Statistical  Service  at  Iowa  State  University 
1931-1947  -  professor,  mathematics,  Iowa  State  University 
1931  -  first  degree  in  statistics  at  Iowa  State  University,  an  M.S.,  was 
awarded  through  the  Department  of  Mathematics  to  his  student, 
Gertrude  M.  Cox, 
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1931  -  brought  R.  A.  Rlahar  to  Iowa  State  Unlvaraity  for  tha  first  time, 
ac  a  vieidnj  profeaaor  of  statistic*  during  the  summer 
1933  -  organlaad  tha  Statistical  Laboratory  as  an  lnstltuta  under  tha 
president's  office 

1933-19*7  -  director,  Statistical  Laboratory,  Iowa  State  Unlvaraity 
1933  -  appointed  first  Station  Statistician  of  the  Iowa  Agricultural 
Experiment  Station 

193*  -  Analysis  of  Variance  published  by  the  Iowa  Stata  University  frees 
1933  -  naked  head  of  the  newly  created  statistical  section  of  the 
Agricultural  Exparlaant  Station 

1933  -  chairman  of  the  Osborne  Club  (local  staff  research  club) 

1936  -  established  the  custom  of  tea-drlnklng  at  a  weekly  statistics 

seminar  which  provided  an  opportunity  for  dialogue  between  staff 
and  students  on  research  problems  presented  by  persons  in  substantive 

areas 

1936  -  president  of  Sigma  XI  (Iowa  State  University  Chapter) 

1937  -  first  edition  of  Statistical  Methods  published  by  Iowa  State  Univeisity 

Press 

1938  -  second  edition  of  Statistical  Methods  published  by  Iowa  State  University 

Prate 

1938  -  established  a  cooperative  agreement  between  the  Statistical  Labora¬ 
tory  and  the  United  States  Department  of  Agriculture  which  provided 
funds  for  the  expansion  of  Laboratory  personnel  and  projects 

1938  -  brought  W.  C.  Cochran  to  Iowa  State  University  aa  a  visiting  professor- 

he  joined  the  staff  In  1939;  Charles  P.  Wlnsor  joined  tha  staff 

1939  -  the  Statistical  Laboratory  was  moved  into  the  newly  built  Service 

Building  (now  known  as  Snedecor  Hall) 

1939  -  elected  Fellow,  American  Statistical  Association 
19*0  -  third  edition  of  Scat la deal  Methods  published  by  Iowa  State 
University  Press 

19*1  -  guest  lecturer.  North  Carolina  Institute  of  Statistics,  summer 

19*2  -  elected  Fellow,  Institute  of  Mathematical  Statistics 

19*3  -  established  a  cooperative  agreement  between  the  Statistical 

Laboratory  and  the  Bureau  of  the  Census;  Hester  Sample  Project 
began 

19**  -  instituted  a  contractural  project  with  t^e  Unites  States  Weather 
Bureau 

19*3-1938  -  editor  of  the  Queries  Section  of  Biometrics 
19*6  -  guest  lecturer,  North  Carolina  Institute  of  Statistics,  summer 
19*6  -  fourth  edition  of  Statistical  Methods  published  by  Iowa  State 
University  Press 

19*7  -  retired  as  director  of  the  Statistical  Laboratory,  Towa  State  University 
19*7  -  Department  of  Statistics  was  organised  at  Iowa  State  University 
19*7-1938  -  professor,  statistics,  Iowa  State  University 
19*7  -  vice  preaidant,  American  Statistical  Association 
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1947  -  guest  lecturer,  Stetlstlcel  Summer  Station,  Virginia  Polytechnic 

Inadtuta 

1948  -  vlaitlng  raaaarch  profaaaor  of  atatlatica,  Alabama  Polytachnlc 

Inatltuta,  spring 

1948  -  president,  Amarlcan  Statistical  Association 

1948  -  Introduced  a  plonaaring  basic  Introductory  course  sequence  In 

statistics  for  undergraduates  at  Iowa  State  University,  especially 
designed  for  students  majoring  In  natural  and  social  sclencea 

1949  -  guest  lecturer.  North  Carolina  Institute  of  Statistics,  swear 

1950  -  limited  first  edition  of  Everyday  Statistics  published  by  William 

C.  Brown,  Dubuque 

1951  -  limited  second  edition  of  Everyday  Statistics  published  by  William 

C,  Brown,  Dubuque  '  * 

1951  -  guest  lecturer.  North  Carolina  Institute  of  Statistics,  summer 

1953  -  consultant  in  experimental  statistics,  Alabama  Polytechnic  Institute 

and  Florida  University,  January-June 

1954  -  elected  honorary  Fellow,  British  Royal  Statistical  Society 

1955  -  consultant  in  experimental  statistics,  Woman's  College  of  North 

Carolina,  January-Aprll 

1955  -  awarded  faculty  citation,  Iowa  State  University 

1956  -  consultant  In  agricultural  statistics,  Brasil,  January-June 
1956  -  fifth  edition  of  Statistical  Methods  published  by  Iowa  State 

University  Press  " 

1956  -  received  honorary  D.Sc,  degree,  University  of  North  Carolina 

1957- 1958  -visiting  professor.  Department  of  Experimental  Statistics, 

North  Carolina  State  University 
1958  -  received  honorary  D.Sc.  degree,  Iowa  State  University 

1958- 1963  -  consultant  In  experimental  statistics  (part-time),  L'.S.  Navy 

Electronics  Laboratory,  San  Diego 

1959- present  -  professor  emeritus,  Iowa  State  University 

1967  -  sixth  edition  of  Statistical  Methods  published  by  Iowa  State 
University  Press 

1969  -  Service  Building  renamed  Snedecor  Hall;  formal  ceremonies  held 
May  18,  1970 
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ACCEPTANCE  REMARKS  OF  0.  K&4PTH0RNE 
ON  BEHALF  OF  0.  W.  SNRDbOOR 

My  assignment  is  a  little  strange  in  that  In  addition  to  acctptlng  tha 
award  for  Profaaaor  Snedecor,  I  have  to  make  some  appropriate  remarks,  Ae 
I  was  not  sure  what  to  lncluda  In  theae,  t  wrote  to  Profaaaor  Snadacor,  He 
replied,  "I  leave  It  to  you  to  decide  what  la  suitable  to  aay.  I  think  you 
know  as  better  than  I  know  myaelf.  You  won't  sake  any  false  or  silly  claims." 

I  do  not  recall  hearing  an  acceptance  speech  for  an  occasion  similar  to 
tha  present  so  I  have  no  precedent  to  follow, 

I  am  very  glad  to  accept  the  award  for  George  Snedecor.  I  see  George 
Snadacor  as  one  of  a  vary  aalact  group  of  people  whose  lives  had  a  certain 
definite  ala— the  interpretation  of  data.  My  list  of  these  la  as  follows! 
Queielec,  who  was  the  first,  perhaps,  to  apply  ideas  of  accurate  statistical 
description  to  a  wide  variety  of  populations;  Galton,  Weldon,  and  then 
Karl  Pearson,  whose  energy  In  quantifying  populations  was  fabulous;  Raymond 
Pearl  of  the  United  States,  similarly;  R.  A.  Fisher,  who,  I  believe,  viewed 
statistics  primarily  as  a  tool  for  the  Improvement  of  human  knowledge;  and 
Snadacor. 

It  la  Interesting  to  me  that  Snedecor  became  Interested  In  statistics 
before  Fisher  did  his  great  work.  In  fact,  Snadacor  Introduced  statistics  at 
Iowa  State  College  in  1915.  This  Is  surely  remarkable  because  before  that 
time  statistics  was  regarded  almost  entirely  as  a  tool  for  economics.  Unfor¬ 
tunately  it  is  still  so  regarded  at  many  institutions  of  higher  education. 

The  big  public  event  of  Snedacor's  work  was  the  publication  in  1937  of 
Statistical  Methods.  I  rate  this  as  second  in  importance  to  one  book  only, 
Fisher's  Statistical  Methods  for  Research  Workers.  It  is  most  Interesting  to 
note  that  at  this  time  Snedecor  was  already  56.  This  should  surely  encourage 
many  of  us  that  there  la  still  time  in  our  lives  to  do  some  outstanding  action. 

I  note  parenthetically  that  this  book  has  brought  recognition  and  fame  to  the 
statistical  effort  at  lows  State  University,  and  was  partly  Instrumental  In  my 
leaving  the  "Sceptrad  Isle,"  and  joining  Its  Statistical  Laboratory. 

I  am  Indebted  to  Dr.  Malln  ox  the  Institute  for  Scientific  Information  for 
the  following  Information.  This  Institute  prepares  the  very  valuable  Science 
Citation  Index.  The  Book  "Statistical  Methods”  of  G.  W.  Snedecor  was  the  most 
cited  book  in  1961  and  from  J 964  to  1967  (the  last  year  for  which  the  Statistics 
are  available)  was  the  second  moat  cited  book.  The  first  most  cited  book  in 
these  years  was  Methods  of  Ensymology  edited  by  S.  P.  Colowlck  and  N.  0.  Kaplan. 
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The  number  of  citations  of  the  book,  given  to  a*  by  Or.  Malln  tri  also 
interesting! 


1961 

361 

1964 

621 

196S 

684 

1966 

806 

1967 

880 

So  ths  usage  of  Snedecor  also  grew  though  not  nacaasarily  at  tha  same  rata  aa 
tha  scientific  literature.  This  book,  now  with  W.  G.  Cochran  aa  coauthor,  haa 
gone  through  six  editions  since  1937  and  has  becoae  to  a  remarkable  extant,  tha 
"pharmacopoeia"  of  collectors  of  data  in  tha  sciences. 

For  the  past  fifteen  years  or  so,  the  field  of  statistics  has  undergone 
controversies  of  the  deepest  kind.  There  are  workers  in  the  "foundations  of 
statistics"  (a  subject  which  Irvin  Bross  says  should  be  banned  from  profes¬ 
sional  statistical  meetings)  who  appear  to  take  the  view  that  "Statistical 
Methods"  is  utter  nonsense,  and  that  the  people  who  follow  these  methods  are 
stupid  or  have  been  misled  by  proponents  of  the  methods.  These  people  are, 
it  la  Implied,  caught  up  In  mores  of  scientific  procedure  that  have  no  real 
logical  basis,  Just  as  in  past  centuries  peoples  were  totally  Involved  In  the 
worship  of  Homeric  gods  which  determined  modes  of  thought  and  accommodation 
to  the  real  world. 

To  understand  Snedecor's  life  and  work,  it  is  critical,  1  believe,  to 
distinguish  at  least  five  aspects  of  the  processes  of  knowledge: 

(1)  Interaction  with  already  available  data  to  develop  Ideas; 

(11)  Collection  of  new  data  according  to  a  thought-out  plan; 

(ill)  Rational  processes  for  Interacting  with  the  new  data,  and  with 
old  and  new  data  together; 

(iv)  Processes  for  drawing  conclusions; 

(v)  Processes  for  choice  of  action  in  the  face  of  the  existent 
uncertainty. 

The  dilemmas  of  statistical  philosophy  are  not  new.  The  recent  resurgence 
of  Bayesian  processes  Is  due,  I  believe,  to  a  failure  on  the  part  of  some  groups 
to  recognize  that  even  a  very  crude  categorization  of  prccesses  of  human 
knowledge  must  Include  all  the  aspects  I  mention  above.  This  resurgence  was 
undoubtedly  stimulated  by  an  earlier  effort  t force  the  whole  matter  Into  the 
single  last  aspect  I  mention. 

Spnaklng  now  as  George  Snedecor,  I  would  say  the  following: 
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Tht  ata  of  ay  Hf«  and  writing*  haa  baan  to  take  part  In  and  to  con- 
tribute  to  the  proceaaaa  by  which  the  human  Bind  approachee  altuatlona  of 
Ignorance!  end  to  develop,  teach  and  practice  raaeonable  waye  to  collect 
data  and  uaeful  waya  to  Interact  with  data,  the  naterlal  of  ny  book 
"Statlatlcel  Mathoda"  ehould  not  be  regarded  aa  giving  final  conclualve 
anawera.  Xt  la  auggeetlve  but  not  axhauatlve.  It  la  rather  an  outline 
of  how  a  rational  Bind  can  approach  a  aituatlon  of  partial  knowledge  and 
partial  Ignorance  In  a  aearch  for  the  truth,  which  le  the  aaaentlal  but 
not  totally  achievable  goal  of  human  endeavor.  ^ 

i  -  * 
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A  STATISTICAL  HIERARCHICAL  MODEL 
FOR  FLIGHT  TEST  DATA  OF  A 
VHF-FN  DISTANCE  MEASURING  SYSTEM  (DUE) 

Erwin  Blast  and  Eddie  Cornelioua 
Avlonlca  Laboratory 
U.S.  Army  Blactronlca  Command 
Fort  Monmouth,  New  Jeraay 

ABSTRACT 

Thia  paper  deacrlbea  a  statistical  hierarchical  model  of  flight  teat  data 
of  a  VHF-FM  dlatance  meaaurlng  modification  kit  for  conventional  communlcatlona 
FN  tranacalvera.  The  dlatance  meaaurlng  equipment  modification  kit  conaleta 
baalcally  of  three  aaparate  unltat  an  attachment  which  connects  an  exlatlng 
airborne  VKP-FM  tranacelver  to  a  ranging  system;  an  attachment  which  converta 
an  exlatlng  ground  VHF-FM  tranacelver  to  a  ranging  tranapondert  and  a  dlaplay 
unit. 

The  analyala  of  the  data  la  to  determine  the  accuracy  of  the  ayatem  under 


dynamic  conditions. 
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I.  OBJECTIVE! 


This  paper  describes  a  statistical  hierarchical  model  for  flight  test 
data  of  a  VHF-FM  Distance  Measuring  equipment  (IME)  Modification  Kit  for  con¬ 
ventional  communication  fm  transceivers. 

It.  DESCItl  PTIONt 

The  Distance  Measuring  Squipiuent  Modification  Kit  consists  of  three  sepa¬ 
rate  unitsi  an  attachment  which  converts  an  existing  aircraft  VHF/FM  trans¬ 
ceiver  to  a  ranging  system)  an  attachment  which  converts  an  existing  ground 
VKF/FM  transceiver  to  a  ranging  transponder)  and  a  display  unit  which  displays 
the  range  between  the  two  transceivers  in  kilometers.  There  are  no  internal 
modifications  made  to  either  of  the  transceivers .  The  modification  can  be 
applied  through  the  external  connections  of  the  existing  transceiver  without 
the  need  of  any  disassembly  of  either  unit. 

The  aircraft  units  have  a  built-in  homing  capability  with  which  they  can 
determine  when  the  aircraft  is  pointed  directly  toward  or  away  from  a  ground 
beacon  which  it  is  receiving.  In  many  applications,  particularly  in  environ¬ 
ments  where  visibility  to  the  ground  is  restricted,  such  as  nighttime  opera¬ 
tion  or  operation  in  heavy  foliage,  it  is  also  desirable  for  the  aircraft  to 
have  an  indication  of  the  range  from  its  position  to  the  beacon  (transmitter) 
on  the  ground  (fig.  1).  The  DME  modification  kit,  in  conjunction  with  an  air¬ 
borne  and  ground  transceiver,  will  provide  the  ranging  function.  By  modifying 
the  transmitted  signal  which  is  normally  used  to  carry  voice,  the  propagation 
delay  and,  therefore,  the  range  can  be  measured  using  such  equipment  as  the 
AN/ARC-131  and  AN/FRC-77.  In  this  way  no  additional  ground  or  aircraft  trans¬ 
mitters,  receivers,  antennas,  etc.,  are  required  and  only  a  small  additional 
attachment  and  some  cabling  need  be  carried.  An  advanced  development  model 
(feasibility  type)  was  developed  by  RCA  for  USABCCM.  The  flight  testing  of  the 
system  was  conducted  jointly  by  USAECCM  and  RCA. 

III.  FUNCTIONAL  DESCRIPTION! 

The  airborne  EME  initiates  a  ranging  cycle  by  using  the  transmit  divider 
to  phase-modulate  the  transmitter  output  signal  through  a  phase  shift  keyer 
inserted  in  the  antenna  line  (see  figure  2),  After  the  end  of  the  airborne 
transmit  period,  the  airborne  transmit  frequency  divider  acts  as  a  phase 
memory  by  continuing  to  count  at  its  normal  rate,  thereby  generating  a  phase- 
coherent  reference  signal.  The  ground  EME  receives  the  transmitted  phase- 
modulated  signal  delayed  in  time  by  an  amount  equal  to  the  one-way  range 
propagation  time  and  locks  the  ground  receive  divider  in-phase  with  the  re¬ 
ceived  signal.  The  ground  receiver  is  then  turned  off.  This  output  is  used  to 
phase-modulate  the  output  of  the  ground  transmitter.  Then,  a  phase-coherent 
signal  is  retransmitted  to  the  airborne  DiE  system.  After  propagation  delay, 
the  airborne  unit  receives  this  3ignnl  and  by  means  of  its  phase-locked  loop, 
locks  its  receive  divider  in-phase  with  the  received  signal.  The  modulation 
frequency  (1.5  kHz)  was  chosen  such  that  its  period  is  equal  to  the  maximum 
EME  round  trip  range  time  (maximum  range  of  the  system  is  100  k-  > .  Since  the 
transmit  divider  is  generating  a  phase-coherent  reference  signal,  range  may 
be  determined  by  measuring  Ll.c  phase  delay  between  the  airborne  receive  and 
transmit  divider  outputs. 

The  phase  keyer  provides  a  periodic  phase  shirt  of  90  degrees  in  the 
transmitter  carrier  which  is  unmodulated,  and  a  -90  degrees  phase  shift  at  the 
receiving  end. 
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Thu  transceivers  associated  with  the  EME  operate  only  in  simplex  modes; 
that  is,  they  cannot  transmit  and  receive  simultaneously.  Therefore,  it  was 
necessary  to  set  up  a  timing  cycle  that  would  enable  the  ground  unit  to  re¬ 
spond  at  the  tine  the  airborne  transceiver  is  in  the  receive  mode.  A  timing 
cycle  for  the  airborne  unit  consists  of  three  periods;  transmit,  switchover, 
and  receive;  and,  conversely,  the  ground  unit  must  cycle  through  receive, 
switchover,  and  transmit.  During  the  switchover  phase  of  this  ,-ycle  the  EME 
ground  unit  is  not  range  tracking;  it  is  working  on  memory.  The  airborne  unit 
must  remember  the  phase  of  the  signal  il  has  transmitted  while  the  ground  unit 
must  remember  the  phase  of  the  signal  it  has  received.  In  addition,  the  air¬ 
craft  is  moving  and  the  range  is  changing. 

IV.  OBJECTIVE  OF  D>1E  TEST  PROGRAM i 

The  tests  were  designed  to  show  to  what  extent  the  equipment  meets  the 
goals  and  requirements  set  forth  in  the  specifications.  Three  types  of  test 
were  conducted;  static  (tower  tests),  hover,  and  dyn  mic  (flight  tests). 

A.  Tower  Tests  (Static).  The  tower  tests  were  c<  nducted  for  the  pur¬ 
pose  of  determining  slant  range  accuracy  under  static  conditions  and,  to 
obtain  some  data  on  phenomena  such  as  interfering  signals,  signal  fading,  and 
multipath.  Furthermore,  the  tower  tests  were  used  in  debugging  the  equipment 
prior  to  flight  testing.  These  tests  made  use  of  Radio  Set*  AN/PRC-77  and 
AN/ARC-131.  The  ground  EME  system  was  installed  on  top  of  a  400-foot  tower 
and  the  airborne  D-IE  system  was  installed  in  an  S-1S3A)  equipment  shelter 
mounted  on  a  3/4-ton  truck.  The  airborne  system  wa»  moved  to  various  sites  of 
a  known  distance  from  the  tower.  The  EME  airborne  system  measured  slant  ^ange 
to  the  tower.  This  data  is  compared  with  the  actual  slant  range  computed  from 
US  C&G  markers  given  in  x-y  coordinates,  and  the  known  height  of  tower  and  the 
elevation  of  each  site.  Test  results  are  shown  in  table  1. 

Table  1.  Static  Test 


(To  calibrate  system) 


sm 

DME  MEASURED 
SLANT  RANGE 
(ou-an)  (km) 

ERROR 

(meters/ 

SAMPLE 

SIZE 

STD  DEV 

FREQUENCY 

(MHz) 

-  se 

.03741 

30.5 

-  72 

.0389 

36.3 

1.89 

1.699 

♦  S 

40 

.0407 

41.6 

2.108 

+218 

.03  56 

46.7 

6.97 

♦  120 

.0428 

30.5 

6.82 

-  20 

.0332 

36.3 

6.B5 

C  .94 

+  90 

40 

.0529 

41.6 

7.02 

+  170 

.1570 

46.7 
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D.  Hover  Tost.  In  order  to  check  the  system  in  an  aircraft  e-. viranraent, 
hover  tests  were  performed.  The  grounJ  unit  was  placed  at  known  x-y  coordi¬ 
nates  and  the  aircraft  hovered  at  several  altitudes  over  a  'known  location. 

The  computed  slant  range  is  compared  with  the  HME  measured  slant  range,  litis 
was  performed  both  on  a  calibrated  channel  and  on  an  uncalibrated  channel. 

The  results  of  this  test  are  indicated  in  table  2. 


Table  2.  Hover  Test 


ALTITUDE 

DISTANCE  TO 
GROUND  SITE 

MEAN:  X 

STD  DEV 

SAMPLE 

SIZE 

FREQUENCY 

(MHz) 

160 

160 

54.687 

16.211 

32 

49.7 

320 

2679 

29.166 

41.173 

24 

40.0 

It  was  concluded  from  the  results  of  both  the  tower  and  hover  testa  that 
the  system  was  operating  properly.  Therefore,  this  brings  us  to  the  most 
important  test,  the  dynamic  test  to  which  the  nested  model  is  applied. 

C.  Dynamic  Flight  Tests.  The  flight  tests  are  used  to  evaluate  the  per¬ 
formance  of  the  equipment  under  dynamic  conditions  where  appreciable  veloci¬ 
ties  and  accelerations  exist.  A  UH-1D  helicopter  was  used  for  this  test.  The 
airborne  unit  was  installed  aboard  the  aircraft  and  the  ground  system  was  lo¬ 
cated  at  a  known  (surveyed)  location  where  an  M-33  Fire  Control  Radar  System 
was  also  located.  The  radar  system  is  used  as  a  reference  standard  for  mea¬ 
suring  the  accuracies  of  the  DME.  The  ground  system  was  located  30  meters 
north  of  the  radar  system  in  order  to  minimise  >»>e  effect  of  rf  interference. 

In  this  test,  the  flight  paths  were  chosen  so  that  the  aircraft  flew 
toward  end  away  from  the  ground  EME  unit  in  a  southward-northward  direction. 
This  flight  path  was  chosen  because  of  the  location  of  the  airborne  antenna. 

As  indicated  in  figure  3,  the  airborne  communication  antenna  is  located  in 
the  tail  section  of  the  aircraft.  Therefore,  the  data  was  grouped  according 
to  direction  (toward  and  away  from  the  ground  IME  unit).  This  test  was  per¬ 
formed  for  nine  frequencies  where  data  was  recorded  at  five  distances  from 
the  ground  fME  unit.  See  tables  3  and  4. 

In  this  test  we  are  dealing  with  a  situation  involving  sampling  within 
each  of  several  major  groups.  To  put  it  correctly,  the  directions  (south  and 
north)  are  two  levels  of  one  (qualitative)  major  factor,  direction.  The  other 
factor,  frequency,  is  nested  (or  subsumed  under)  within  each  level  of  the 
major  group.  Thus,  direction  is  a  major  classification  and  frequency  is  a 
minor  classification. 

There  are  two  major  groups,  south  and  north  directions.  The  frequencies 
are  nested  within  the  directions.  A  hierarchical  model  is  characterized  by 
the  fact  that  the  levels  of  one  factor  (minor  classification)  are  nested 
within  each  of  the  levels  of  another  factor  (major  classification). 

It  is  well  known  that  the  number  of  levels  of  the  nested  factor  (fre¬ 
quency  in  this  experiment)  need  not  be  the  same  for  all  levels  ot'  the  major 
lector.  We  assume  that  the  frequencies  are  selected  randomly,  i.  e.,  fre¬ 
quency  is  a  random  factor.  The  directions  are  also  considered  random.  Our 
model  is  given  by  the  following  structure. 
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Figure  3. 


Table  3.  Test  Frequencies  for  Southward  and  Northward  Flights 


SOUTHWARD  FLIGHT 

NORTHWARD  FLIGHT 

F^  =  49.7  MHz 

P6  =  46.7  MHz 

F2  *  42.9  MHz 

F7  =  41,6  MHz 

F3  =  38.1  MHz 

Fs  =  36.3  MHz 

F4  =  34.1  MHz 

r  9  =  32.1  MHz 

F5  =  30.1  MHz 


Table  4.  Distance  (D)  from  Radar  (M-33)  to  Aircraft  in  Kilometers  (kra) 


TEST  POINTS 

1 

Df  *  7.74 

2 

Dj  ■  4.32 

3 

D3  »  1.09 

4 

D4  =  3.52 

5 

D5  =  6.81 

V.  THE  STRUCTURE  OF  THE  MODEL 

xijk  *  vi  ♦  At  ♦  ® j(i)  ♦  «k(ij> 

The  symbolx  Xijk  and  e|c(tj)  are  defined  in  the  sequel  i  *  1,  2;  j  «  1, 
2,  ....  5  for  the  B’s  in  A*;  and  i  *  6,  7,  8,  9  for  the  B's  in  A2* 

Aj  a  South  (S)  i  Aj  a  North  (N} 

The  B'a  are  the  levels  of  frequency  nested  under  the  respective  levela 
of  directions,  north  and  south. 

This  model  has  no  interactions  since  the  frequencies  are  not  crossed 
with  the  directions. 

u  »  the  population  mean 

A£  =  the  effects  associated  with  the  major  groups,  (i  =  1,  2) 
u  is  defined  so  that  -- 

1  \  “  0 
ial 

Bj(i)  =  the  fluctuations  between  the  J  random  samples  drawn  from  each 
of  the  1  major  groups  (I  *  2), 

The  A^  are  so  defined  that  -- 
J 

£  Bj(i)  =  0 
j’l 

eijk  *  random  errors.  They  are  assumed  to  be  normally  distributed 

with  the  variance  equal  to  O 


“Hie  Bj(l)  are  so  defined  that  =  0  for  any  ij.  Denote  the  esti¬ 

mates  of  u,  A^ ,  Bj(q,  and  e^(ij)  byO,£^f^j(i)  and  e^^j)  respectively. 
Denote  the  sample  unbiased  estimates  of  u,  Aj_ ,  Bj(Q  by  STj  -  57)  and 
(xij  -  respectively.  For  each  i,  we  obta'.n: 

I  1 

2  aL  "  2  (*i  *  *>  *  0 

l»l  i"l 

J 


This  shows  that  the  total  sum  of  squares  is  equal  to  the  sum  of  squares 
between  the  levels  of  A,  plus  the  sum  of  squares  between  the  levels  of  B 
nested  within  each  level  of  A,  plus  the  sum  of  squares  of  the  errors.  See 
table  6. 

SSBM  8  GUm  of  squares  between  major  groups. 

SSf)n>  3  sum  of  squares  between  minor  groups  (within  major  groups). 

SSwm  -  sum  of  squares  within  minor  groups. 

S~BM  s  NL  <?t  -  *>2  =  ^Ntmt2 

where  m^  5  (x^  -  x)  is  the  sample  estimate  of  the  major  group  effect 

SSBM  ^  nfj  (xy  -  Tl)  *  ^^nljHj2 

where  tij=(xj_j  -  5T^)  is  the  sample  estimate  of  the  minor  group  effect 


Table  5.  A  Two-Factor  HicrarchLcal  (Nested)  Tabic  for  EME  Experiment 
(Entries  arc  errors  (in  meters)!  EME  Slant  Range-Radar  Range.) 


Direction 

S 

■ . . . "  1 

_ N _ 1 

Frequencies 

Fl 

F2 

F3 

f4 

m 

F6 

F7 

H 

F? 

Distance  from 
Ground  Station 

190 

30 

uo 

-  50 

490 

-270 

300 

-400 

-390 

d2 

180 

-  20 

230 

-  180 

350 

80 

O 

O' 

1 

- 

90 

d3 

140 

*  60 

60 

-  180 

310 

-160 

-no 

-240 

70 

DU 

100 

-130 

140 

-  420 

210 

-  60 

-390 

-210 

-130 

DS 

130 

-  60 

-100 

-  470 

560 

ISO 

-4  50 

-100 

-170 

Frequency 

Total  s 

7b0 

-120 

-370 

-1300 

1920 

-230 

-740 

-950 

-530 

Sample  Size 

5 

5 

S 

S 

5 

5 

5 

5 

5 

Direction 

Total 

♦1610 

-2450 

Tabic  6.  General  Analysis  of  Variance  Table  for  a  Nested  Experimental  Model 


SOURCE  OF 
VARIATION 

DEGREE  OF 
FREEDOM 

SUM  OF  SQUARES:  SS 

Ai 

1-1 

y  li!  _ 

4*  KJ  ”  KIJ 

^A/l-l 

Bj(i) 

m 

I  J  J 

y  y  sul  -  Y  V 

i  j  K  i  KJ 

^BAO-l) 

ekUj> 

IJ(k-l) 

I  J  K  I  J 

SSI-VS  I ai 

i  j  k  i  j  K 

SSeAJ(k-l) 

I  J  K 


Totaisp;(iJK  -  i);  =  2  2  2  x2ijK  — 

i  j  k  KIJ 


(xijk  "  *  X  X 

i  j  k 


®ijk 


where  eljk  «  (xt  jk  -  ytj) 

Let  CF  (correction  factor)  ■  C  * 


the  total  magnitude  of  all  the  observations,  and 
;  N  *  the  grand  total  number  of  observations  of 


the  experiment. 

The  sum  of  squares  for  the  i^  direction  is: 


DSS 


)/JK 


-  C 


In  this  experiment  one  major  r.roup  (South)  has  five  minor  groups 
(frequencies),  with  five  observations  for  each  frequency.  The  other  major 
group  (North)  has  four  minor  groups  with  five  observations  for  each  of  three 
frequencies  and  four  observations  for  one  frequency  (Fg). 

The  sum  of  squares  for  the  directions  (Dgg)  is: 

°SS  *  [<Tl2>/2S  ♦  (T2)?A9]  -  C 

where  Ti  and  T2  are  the  magnitudes  of  the  total  number  of  observations  of 
the  major  groups  1  and  2  respectively. 

n*  the  analysis  of  vari«::ce  for  hierarchical  classification 

a.  For  the  total  sum  of  squares,  the  degrees  of  freedom  are  44  -  1  =  43. 

b.  For  the  direction  sum  of  squares,  the  degrees  of  freedom  are  2-1  •  1. 

c.  For  the  frequency  s»lra  of  squares,  the  degrees  of  freedom  are  the 

number  of  frequencies  minus  the  number  of  directions:  9  -  2  *  7. 

d.  For  the  error  sum  of  squares,  the  degrees  of  freedom  are  the  number 
of  observations  minus  the  number  of  frequencies:  44  -  9  *  35. 

Let  us  examine  the  analysis  of  variance  tabic  and  analyze  the  F -test 
f°r  significance  of  various  sources  of  variation.  We  compare  the  frequency 
mean  square  with  the  error  mean  square: 

F  -  169539^972  =  6>659 
25457.57 

The  tabular  value  for  7  and  35  degrees  of  freedom  at  the  5  percent  significance 
level  is: 
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p7*  3S;  05  =  2*29 

Vj,  jj,  *  3.20  (at  the  1  percent  level) 

In  cither  case  there  is  a  difference  between  the  frequencies  (not  highly 
significant) . 

For  the  significance  of  directions,  we  compare  the  direction  mean  square 
with  the  error  me:;::  squares 

p  a  ft2I££i&&£2  >  15.852 
25457.57 

The  tabular  values  for  1  and  3S  degrees  of  freedom  ares 

Fl»  35;  05  "  4*12 
pl»  35;  01  "  7*42 

There  is  a  significant  difference  between  direction  (south  and  north).  The 
observations  (the  errors)  vary  both  from  frequency  to  frequency  and  from 
direction  to  direction. 


VII.  FORMULAS  FOR  HIERARCHICAL  MODEL 

n^j  ■  number  of  observations  in  jth  minor  group  (frequencies)  of  the  1th 
major  group  (direction) 


■  size  of  jth  minor  group 


Ni 


th 


nij  •  number  of  observations  in  the  i  major  group 
L 

size  of  major  group 
I  I  J 

y  Hi  e  ^  ^  n  ■  Nj  «■  Nj  ■  grand  total  number  of  observations 


i«l  j»l 


xiik  "  the  kth  individual  observation  in  the  jth  minor  group  of  the 
J  ith  major  group 

*  a  ^x^/n^j  *  Ti j/nt j  “  »ean  valui  of  observations  of  jLh  subgroup 


1  i 


(minor  groups  of  Ith  major  group 


Tjj  •  total  value  of  k  observations  of  jth  subgroup  of  it}*  n.i  j.'i  >\< 


where  = 


Ujk 


■  the  total  value  of  th.-  ,-!>■  ,  ,  x 


jth  subgroup  in  the  i1^  major  group 


the  total  value  of  the  observations  of  the  i**1 
major  group 


J 


*  the  total  value  of  the  observations  in  the  1 
major  group 

■  Tt  «  grand  mean  oE  the  entire  set  of  observations 
T  ■  X. . .  •  T/N 

VIII.  CONCLUSION 

In  this  experiment  one  major  group  (South)  has  five  minor  groups  (fre¬ 
quencies)  with  five  observations  for  each  frequency.  The  other  major  group 
(North)  has  four  min'T  groups  with  five  observations  for  each  three  fre¬ 
quencies  . 

Let  us  examine  the  analysis  of  the  variance  table  and  analyse  the  P-test 
for  significance  of  various  sources  of  variation.  It  is  found  that  there  is 
e  difference  betwee-  frequencies  (not  highly  significant)  and  there  is  a 
significant  difference  between  direction  North  and  South.  Therefore,  the  sys¬ 
tem  varies  both  from  frequency  to  frequency  and  from  direction  to  direction. 

The  errors  associated  with  the  frequencies  are  due  to  the  instability  of 
the  transceivers.  It  was  determined  during  laboratory  testing  that  when  the 
system  was  calibrated  on  one  frequency,  this  calioration  did  not  hold  for  all 
frequencies  in  the  30  to  70  MH*  bend. 

When  the  data  was  collected  for  the  North  direction,  the  aircraft  was 
going  away  from  the  ground  transponder  and  the  data  for  the  South  direction 
was  collected  when  the  aircraft  was  going  toward  the  ground  transponder. 
Therefore  from  figure  1  it  can  be  seen  that  the  communications  antenna  (IWE 
used)  fs  looted  in  the  tail  of  the  aircraft.  It  may  be  concluded  that  the 
orientation  of  the  aircraft  influences  the  antenna  pattern  and  received  and 
transmitted  signal  levels.  Therefore  the  decrease  in  signal  level  when  the 
aircraft  was  going  toward  the  transponder  caused  more  jitter  in  the  system. 
The  results  are  indicated  in  table  S  where  the  mean  is  168.4  meters  for  the 
S-Fraq  and  -134.375  meters  for  the  N-Frcq. 


Ti  “2.  Xl-' 

J-l 

I 

T,x...  -  I  Tl 


i*l 


Table  7.  Mean  and  Standard  Deviation  for  the  Teat  Frequencies  F,  Through  F9 


Tabl/  9.  The  Analysis  o'  Variance  for  HierarchLc.nl  Classification  of  Data 


SOURCE  OF 
VARIATION 

SOURCE  OK 
VARIATION 

df 

SS 

MSS 

F-ratioa 

Between  dir¬ 
ections 

Between  major 
groups 

1 

403568.69 

40356P..69 

IS. 85 

Between  fre¬ 
quencies 

Between  minor 
(within  sajor) 
groups 

7 

1186779. 9‘J 

169539.99 

6.66 

Error  within 
frequencies 

Error  within 
minor  groups 

35 

891015.00 

25457.57 

TOTAL 

43 

2481363.64 

57706.13 

Table  10,  Standard  Deviations  from  Analysis  of  Variance 


SD 

SF 

SE 

ST 

635.270 

411.752 

159.554 

240.221 

Wheret 


sD 

■ 

CMS, 

DMS 

is 

the 

Sp 

■ 

FMS, 

FMS 

is 

the 

®E 

a 

IMS, 

EMS 

is 

the 

S  j 

■ 

IMS, 

IMS 

la 

the 

Direction  Mean  Square 
Frequency  Mean  Square 
Error  Mean  Square 
Total  Mean  Square 
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ABSTRACT 

A  multivariate  model  for  a  design  of  experiments  to  obtain  Infor¬ 
mation  for  teatlng  the  Semiautomatic  Flight  Operations  Center  (SAFOC) 
for  air  traffic  control  la  presented. 


A  scheme  for  ranking  four  different  methods  of  operation  to  deter¬ 
mine  the  beet  method  of  operating  tha  system  is  discussed  using  confidence 
intervela  conetructed  from  Bonferroni's  inequalities. 


The  interactlona  among  controller  teema,  traffic  density  levels,  and 
methods  of  operation  are  discussed.  Tests  for  significance  of  interactions 
are  presented. 


An  alternative  ranking  schema  designed  to  handle  traffic  densities 
levels  lntaractlona  is  also  presented. 


The  remainder  of  this  article  has  been  reproduced  photographically  from 
tho  author's  manuscript. 
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A  MULTI  VARIANT  STATISTICAL  MODEL  FOR  A  SEMIAUTOMATIC 
FUGHT  OPERATIONS  CENTER  (SAFOC) 


1  INTRODUCTION: 

1.1  OBJECTIVES 

The  purpose  of  this  paper  is  to  tttaouaa  the  design  of  experiments  for  testing  the 
performance  and  capabilities  of  a  Semiautomatic  Flight  Operations  Center  (SAFOC). 

The  general  purpose  of  this  series  of  tests  is  to  determine  the  effectiveness  of 
SAFOC  in  performing  its  mission  in  a  realistic  environment.  The  specific  purpose  of 
the  tests  described  in  this  paper,  denoted  as  Phase  1  tests,  is  to  determine  the  best 
method  for  operating  SAFOC  and  to  find  the  system  performance  measures  and  system 
effectiveness  measures  that  can  be  attained  under  these  optimum  operational  condi¬ 
tions. 

The  Phase  11  test  effort,  not  described  in  this  paper,  will  complete  the  task  of 
evaluating  SAFOC  in  a  realistic  environment  and  will  determine  those  changes  to 
SAFOC  which  will  improve  its  mission  effectiveness. 

1.2  BACKGROUND: 

SAFOC  will  be  tested  and  evaluated  at  the  Federal  Aviation  Agency  (FAA), 

National  Aviation  Facilities  Experimental  Center  (NAFEC),  Atlantic  City,  N.  J. ,  by 
the  C.S.  Army  Electronics  Command  (ECOM),  Avionics  Laboratory  (AMSEL- VL-G), 
Fort  Monmouth,  N.  J.  The  evaluation  testing  of  the  SAFOC  will  be  performed  in  two 
phases  designated  Phase  I  and  Phase  II.  Phase  1  tests  are  scheduled  for  the  period 
from  1  March  through  :tl  August  l’J70.  Phase  II  tests  are  to  be  conducted  between 
1  October  lt»70  and  Jl  January  1071. 

These  tests  arc  designed  to  determine  system  operating  chmactoristics  in  a  real¬ 
istic  environment  for  evaluation  of  system  merits.  This  method  if  testing  is  usually 
termed  a  "Military  Potential"  test.  The  SAFOC  system  was  originally  tested  during 
the  acceptance  lest  by  ECOM  to  determine  whether  the  equipment  satisfied  the  con¬ 
tractual  requirement  described  in  the  systems  description.  These  tests  were  intended 
to  validate  the  design  as  called  out  by  the  specifications. 

The  test  flow  diagrrm,  figure  1,  shows  that  the  SAFOC  system  Is  tested  with  a 
number  of  Army  air  traffic  controller  teams  at  various  air  traffic  densities  for  tne 
purpose  of  giving  the  system  a  fair  test  over  a  broud  spectrum  of  conditions.  The 
SAFOC  equipment  was  designed  for  flexibility  in  operation  and  can  be  operated  in  a 
number  of  ways.  For  this  reason,  various  operational  methods  oi  doctrines  were  in¬ 
corporated  into  the  test  scheme. 
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Figure  1.  SAFOC  Design  of  Experiments,  Test  Flow  Diagram 
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These  tests  of  the  SAFOC  system  will  provide  data  for  analysis  and  evaluation. 

The  evaluation,  as  described  later  In  this  paper,  will  measure  system  effectiveness 
and  variations. 

1. 3  SAFOC  TECHNICAL  OBJECTIVES: 

The  technical  objectives  of  the  SAFOC  are: 

1.  Tt>  regulate  Army  air  traffic  in  an  efficient  and  orderly  manner  under  instru¬ 
ment  flight  rules  (IFR). 

2.  To  provide  flight  following  capability  under  visual  flight  rules  (VFR). 

3.  To  Improve  Information  flow  and  transfer  among  system  elements  and  units 
being  supported. 

4.  To  provide  computer  facilities  which  can  automatically  assimilate  and  ana¬ 
lyze  air  traffic  data  for  display  and  decision- making  by  an  operator. 

5.  To  provide  a  rapid  and  reliable  means  for  making  commanders  aware  of  the 
current  air  traffic  situation  for  integration  into  overall  tactical  planning. 

6.  To  perform  specific  functions  required  in  an  air  traffic  regulation  system, 
including  flight  plan  processing,  flight  following,  position  determination, 
rescue,  traffic  corridor  planning,  and  exchange  of  information  with  other 
control  centers. 
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2.  GENERAL  SYSTEM  DESCRIPTION* 

2. 1  DESCRIPTION  OF  EQUIPMENT 

Figure  2  represents  the  aotuil  SAFOC  configuration  within  the  van.  Included 
within  the  8AFOC  system  are  the  following  subsystems* 

1.  Data  processing  subryatem 

2.  Radar  processing  subsystem 

3.  Display  subsystem 

4.  Manual  backup  subsystem 

Figure  3  Is  a  block  diagram  showing  the  Interconnections  of  the  above  subsys¬ 
tems  within  the  SAFOC  system. 

2. 2  SYSTEM  CAPABILITIES 
2. 2. 1  System  Functions 

The  SAFOC  provides  an  air  traffic  regulation  service  by  collecting,  analysing, 
and  disseminating  the  Information  necessary  to  regulate  the  movement  of  Army  air¬ 
craft  under  instrument  flight  rules  (!FR)  and  monitor  the  movement  of  cooperating 
aircraft  under  visual  flight  rules  (VFR).  The  air  controller(s)  perform  the  air 
traffic  regulation  function  using  data  processing  and  display  equipment  to  provide 
the  following  capabilities: 

a.  Flight  Data  Processing** 

1.  Flight  data  entry-air  filed,  ground  filed 

2.  Flight  plan  clearance 

3.  Flight  plan  activation 

4.  Aircraft  position  determination 

5.  Conflict  prediction 

6.  Collision  avoidance 
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7.  Accommodating  flight  plan  changaa 


8.  Flight  plan  deactivation 

b.  Flight  Following 

c.  Flight  Handoff 

d.  WanOftcatlon  Aaaiatanoe 

e.  Emergency  Assistance 

f.  Air/Ground  Coordination 

g.  Ground/Ground  Coordination 
2.  2. 2  Flight  Tracking  Methods 

SAFOC  provides  five  metHbds  of  flight  tracking  which,  in  order  of  their  assigned 
priority,  are: 

a.  Data  Link  (auto-tracking  only) 

b.  Radar  Beacon  (auto-tracldng) 

c.  Radar  Beacon  (rate-aided  manual  tracking) 

d.  Radar  Skin  Return  (rate-aided  manual  tracking  only) 

e.  Plan  Follow 

2.3  SAFOC  TEST  CON  FIGURATION: 

Figure  4  shows  the  SAFOC  test  operations  and  information  flow  diagram.  As 
illustrated,  the  scenario  generator  program,  prior  to  actual  testing,  generates 
scenarios  and  scripts  based  on  random  processes.  At  the  time  of  testing,  the 
scripts  are  given  to  the  flight  simulator  pilots,  who  keep  comprehensive  logs  of 
all  actions  they  undertake. 

he  pilots  simulate  actual  flights  using  target  generators  which  are  part  of 
NAFEC’s  data  link  simulation. 

Using  a  pr ©-determined  operational-procedural  mode,  SAFOC  controls  the 
simulated  flights  and  produces  exhaustive  time  histories  on  magnetic  tape.  These 
histories  include  all  actions  performed  by  the  equipment  or  by  the  controller. 

The  raw  output  data  tape  and  the  target  generator  history  tape  are  processed 
using  a  aeries  of  formatting  and  editing  programs. 

Preceding  page  blank 
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PRIOR  TO 
PHASE  1  « 


The  processed  tapes  are  then  operated  upon  using  statistical  programs  in  ac¬ 
cordance  with  the  desigo  of  experiments)  providing  the  desired  data  outputs  to  be 
described  later. 


3.  SAFOC  SYSTEM  EVALUATION: 

3.1  OBJECTIVE 

The  purpose  of  the  SAFOC  system  evaluation  to  be  described  is  to  determine 
the  best  operational-procedural  combination  for  SAFOC.  The  tests  will  consist  of 
s  series  of  scenarios  of  three  different  trnfflc  levels.  Each  of  four  controller  teams  will 
operate  the  SAFOC  according  to  four  different  ope-ational-procedural  combinations. 
The  outputs,  consisting  of  system  effectiveness  measures,  will  be  ranked  to  deter¬ 
mine  the  best  operational-procedural  mode. 

3. 2  TEST  FLOW 

Figure  5  illustrates  the  overall  statistical  test  flow  for  Phase  I  testing  of  SAFOC. 
As  shown,  the  ultimate  objective  is  to  determine  the  best  operational-procedural 
mode.  Since  the  ranking  techniques  assume  that  all  controller  teams  are  of  approx¬ 
imately  equal  proficiency,  it  is  first  necessary  to  test  the  validity  of  this  assumption. 
If  it  is  found  that  controller  performances  are  not  uniform,  further  training  and  re¬ 
testing  is  necessary.  The  techniques  for  testing  controller  performance  are  described 
In  paragraph  3.  7. 

When  it  is  found  that  all  controllers  have  been  trained  to  a  level  where  any 
existing  differences  have  no  significant  effects  on  the  system  effectiveness  measures 
(see  paragraph  3. 5),  the  controller  team  tests  are  begun  on  the  SAFOC  system.  The 
performance  and  effectiveness  measures  generated  from  these  testa  will  be  used  to 
evaluate  SAFOC  capability  and  to  determine  the  best  operational-procedural  combina¬ 
tion. 


3.3  STATISTICAL  FACTORIAL  DESIGN  MATHEMATICAL  MODEL: 

The  basic  model  for  this  test  is  used  In  all  48  runs  of  the  series.  The  model 
assumes  that  the  controller  teams  have  essentially  the  same  performance  levels. 

(The  validity  of  this  assumption  Is  to  be  checked  prior  to  the  Initiation  of  record 
runs  according  to  the  teats  described  in  3.7.)  The  effects  of  random,  or  possibly 
even  systematic,  changes  In  performance  among  teams  and  from  day-to-day  is 
minimized  by  randomizing  the  order  of  performance  of  the  48  test  runs. 

The  purpose  of  the  initial  experimental  design  is  to  determine  the  variation  of 
the  dependent  variables  (in  the  mean)  with  the  independent  variables  and  to  rank  these 
outputs.  Further,  the  purpose  is  to  determine  what  sort  of  functional  dependencies 
may  exist  between  lnpi  ts  and  outputs.  When  such  dependencies  are  discovered.  It 
may  he  possible  to  perform  regression  analysis,  using  an  appropriate  model,  to 
obtain  analytical  expressions  for  any  empirical  variations. 
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Figure  5.  Flow  Diagram  of  Phase  I  Statlstica 


Reference  runs  will  be  generated,  in  which  aircraft  will  enter  the  system  and  fly 
through  without  any  air  traffic  control  or  regard  for  conflicts,  entrance  rates  (gener¬ 
ated  by  a  random  number  process  on  a  computer!  will  be  controlled  and  changed  from 
run  to  run  as  an  Independent  variable.  The  entire  set  of  reference  runs  will  be  avail* 
able  as  history.  The  same  "input"  conditions  are  then  used  In  runs  In  which  air  traffic 
control  is  exercised.  Comparisons  between  the  "flyth rough"  and  the  controlled  runs 
can  that  indicate  the  effectiveness  of  the  SAFOC  method  of  operation. 

Reference  runs  will  consist  of  the  flight  path  charts  and  scripts  from  the  AEL 
scenario  generator.  These  contain  the  time  required  for  a  Right  to  begin  and  end 
the  mission  and  represent  the  shortest  time  possible  for  a  flight  to  be  in  the  system. 
Control  functions  performed  on  the  flight  may  require  additional  flight  time.  Thus, 
the  reference  runs  represent  an  orderly  account  of  operation  under  "ideal"  conditions 
as  n  start  towards  understanding  performance  under  "Held"  conditions. 

3.4  INPUT  TEST  PARAMETERS: 


Four  methods  of  operating  the  SAFOC  System  are  defined  as  follows: 

a.  Oj.  Operate  the  system  with  two  adjoining  ATC  sectors,  one  at  each  display. 
Each  controller  Is  then  independently  responsible  for  his  own  sector. 

b.  O2.  Operate  the  system  with  one  sector,  of  the  same  area  as  in  Oj,  and 
divide  the  various  ATC  functions  between  the  two  displays.  One  controller  Is 
the  senior;  the  other  is  the  junior. 

c.  Pj.  An  active  procedural  mode  where  all  flights  are  displayed  to  the 
operators. 

d.  P2.  A  passive  procedural  mode  with  displays  only  as  a  result  of  alerts  to 
which  the  operator  reacts. 

From  the  above  we  generate  four  operational  concepts,  namely: 


°ipr 


°ip2.  o2p4. 


O  P 
2  2 


These  are  the  four  methods  which  will  be  tested  by  the  various  performance  measures. 


Three  levels  of  traffic  will  be  generated  to  test  the  four  methods.  Four  teams  of 
two  controllers  each  will  operate  the  SAFOC  for  each  method  at  each  traffic  level. 
Figure  6  shows  the  interactions  of  the  test  inputs.  Table  1  gives  the  experimental 
design  in  the  notation  to  be  used  hereafter. 


The  order  of  performance  of  the  test  runs  is  randomized  as  shown  in  Table  2 
to  eliminate  the  effects  of  the  learning  process  of  the  operators  in  the  course  of 
performing  the  test  runs. 
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Figure  0.  Experimental  Design 


able  1.  EXPERIMENTAL  DESIGN  FOR  THREE  TRAFFIC  LEVELS  AND  FOUR  CONTROLLER  TEAMS 
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Table  2.  ORDER  OF  TE8T  PERFORMANCE 
(Randomised) 
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If  practical  considerations,  such  as  controller  avallabtlity  and/or  equipment 
availability,  require  deviations  from  this  order  of  testing,  attempts  will  be  made  to 
estimate  the  resulting  effects  upon  test  results.  In  such  cases  the  test  plan  will  be 
modified  accordingly. 

Specifically,  If  four  independent  but  equally  proficient  controller  teams  are  not 
available  for  the  test  duration,  not  enough  data  will  be  obtained  to  make  significant 
evaluations  of  the  system.  If  necessary,  the  test  plan  will  be  modified  to  test  per¬ 
formance  using  only  one  controller  per  team  and  the  two  sector  mode  of  operation 
will  not  be  uttlieed. 

3.5  OUTPUT  TEST  PARAMETERS: 

This  section  describes  the  system  performance  measures  and  system  effective¬ 
ness  measures  which  are  obtained  by  reducing  the  data  generated  from  test  runs  on 
SAFOC. 

3.5.1  System  Performance  Measures 

These  are  measures  of  the  individual  Items  which.  In  total,  influence  the  system 
effectiveness.  The  performance  measures  represent,  for  example,  the  actual  con¬ 
tributors  to  total  workload,  and  it  is  through  Improvements  in  the  performance  meas¬ 
ures  that  the  effectiveness  measures  can  be  improved. 

The  following  are  the  system  performance  measures: 

1.  Time  to  perform  a  service  for  each  service 

2.  Service  rate 

3.  Waiting  time  for  service 

4.  Event  time  history 

5.  False  dismissal  probability 

6.  Actual  density  history 

7.  Queue  lengths 

8.  Typewriter  errors 

9.  Near  miss  history 

10.  Communication  time  history 

11.  Number  of  impossible  requests 
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12.  Altitude  change  history 

13.  Closest  approach  history 

Service  time  la  time  between  initiation  of  service  and  completion  of  service. 

The  services  to  be  considered  are: 

1.  T5T>«writer 

2.  Handoff 

3.  Activate 

4.  Clear  flight  plan 

5.  Coordinate 

6.  Conflict  resolution 

7.  Alert  servicing 

(a)  Clearance  alert 

(b)  Conflict  1  alert 

(c)  Conflict  2  alert 
<d)  Coordinate  alert 
(o)  Emergency  alert 

(f)  Flight  overdue  alert 

(g)  Flight  plan  complete  alert 

(h)  Flight  not  active  alert 
(i>  Poor  tracking  alert 
(J)  Handoff  alert 

For  each  service  mentioned  above,  the  service  rate  is  determined  by  dividing  the 
number  of  times  service  is  performed  in  a  given  time  span  by  the  time  span  duration. 

Waiting  time  for  service  is  the  time  between  initiation  of  a  request  for  service 
and  the  initiation  of  that  service. 
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Oteue  length  la  the  number  of  fllghtc  waiting  for  service  at  any  time. 

Falae  dismissal  probability  la  the  number  of  actual  near  miaaea  which  did  not 
regiater  a  conflict  1  or  2  alert  divided  by  the  total  number  of  near  mi8aea. 

Impoaalble  r  ecru  eat  alerts  are  generated  whenever  a  controller  aaks  the  computer 
to  perform  an  Impossible  task.  These  will  be  counted  by  controller. 

Actual  density  history  la  a  time  reoord  of  actual  flight  entrance  and  exits  from 
the  system. 

Near  miss  history  Is  a  record  of  all  flight  pairs  with  actual  separations  less  than 
the  minimum  required. 

Event  time  history  is  a  complete  history  of  all  events  In  order  of  occurrence  for 
each  flight. 

Typewriter  errors  are  counts  of  the  number  of  typing  errors  committed  by  the 
typist. 

Altitude  change  history  records  all  altitude  changes  and  the  time  of  change. 

Closest  approach  history  records  all  flight  pairs  within  a  specified  distance,  the 
closest  approach  of  each  flight  pair  and  the  time  of  closest  approach. 

Communication  time  history  records  each  time  a  communication  is  performed 
and  the  duration  of  the  communication. 

3.5.2  System  Effectiveness  Measures 

The  system  effectiveness  measures  are  used  to  provide  relative  rankings  of  the 
operational  procedural  mixes  and  to  evaluate  relative  controller  team  performance. 
The  following  measures  are  chosen  because  It  Is  believed  they  represent  the  charac¬ 
teristics  most  important  to  the  user: 

a.  Safety 

b.  Controller  Workload 

c.  Communications  Workload 

d.  Delays 

e.  Throughput 

f.  Capacity 

g.  Uncontrolled  Time 
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9»fatv  l»  defined  aa  the  number  of  near  misses  per  aircraft  mile  flown.  This  is 
measured  by  computing  the  number  of  flights  separated  by  lesB  than  the  minimum 
distance  during  a  run  divided  by  the  total  number  of  flight  miles  flown  during  that  run. 
For  the  purpose  of  this  measure,  the  run  begins  when  a  steady  state  density  ie  reached. 

Controller  workload  is  defined  as  total  time  for  all  flight  hooka  in  a  given  time 
span  di\  ided  by  the  time  span.  Controller  workload  can  be  used  to  estimate  controller 
replacement  rates,  controller  scheduling,  etc.  Since  some  tasks  the  controller  must 
perform  may  be  more  taxing  than  others,  workload  will  be  measured  b;  .auk  (handotf, 
conflict  resolution,  etc. )  as  well  as  by  total  time  that  work  of  any  kind  is  done. 

Communications  workload  is  defined  as  the  time  spent  in  communications  during 
a  run  divided  by  the  duration  of  the  run. 

Delays  are  defined  as  the  actual  departure  time  delay  from  the  planned  departure 
time.  Delays  are  believed  to  be  important  in  a  tactical  situation.  It  does  little  good 
to  get  aircraft  safely  to  a  particular  location  if  they  arrive  too  late  to  be  of  use  In 
some  situations.  This  measure  is  easily  computed  by  subtracting  actual  departure 
times  from  the  planned  departure  time. 

Through,  ut  is  defined  as  the  actual  number  of  flights  entered  during  study  state 
divided  by  the  number  of  planned  entries  in  that  time  period.  The  number  of  flights 
entered  in  a  given  time  divided  by  the  number  of  planned  entrances  measures  the 
ability  of  the  system  to  obtain  a  desired  throughput  rate. 

Capacity  is  defined  as  the  peak  flight  density  safely  handled  by  the  system.  This 
is  obtained  by  performing  a  regression  analysis  on  near  misses  vs.  peak  density. 
System  capacity  is  that  peak  density  who  » the  resultant  regression  curve  first 
exhibits  significant  non-zero  slope. 

Uncontrolled  time  is  defined  as  the  total  time  of  flights  within  the  SAFOC  control 
area  without  being  controlled  by  SAFOC. 

J.  6  MATHEMATiC A!  MODEL:  Opprattonal-Procedural  Mode  Ranking 

Figure  7  shows  the  flow  of  statistical  tests  necessary  to  arrive  at  a  ranking  of 
the  four  procedural  modes  (operational-procedural  combinations). 

A  series  of  48  tests,  as  discussed  in  3.4,  will  be  :un  using  the  NAFEC  flight 
simulators  as  input  to  SAFOC.  The  performance  and  effectiveness  measures  listed 
in  3.  r-  will  be  extracted.  The  discussed  in  the  following  sections  will  refer  to 
any  one  of  the  system  effectiveness  measures.  The  four  procedural  modes  will  be 
ranked  according  to  each  effectiveness  measure,  and  the  final  ranking  will  be  per¬ 
formed  as  In  3.6.6. 

Returning  to  figure  7,  it  is  assumed  (as  stated  previously)  that  the  controllers 
have  attained  approximately  equal  levels  of  proficiency.  This  assumption  will  be 
tested  as  described  in  3.7. 
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Figure  7.  Statistical  Testing:  Comparison  of  Operational- Procedural  Combinations 


After  this  assumption  is  validated,  the  first  test  on  the  48  runs  is  a  test  for 
interactions  between  traffic  levels  and  the  procedural  modes.  TTiis  test  is  described 
In  3. 6.1. 


If  significant  interactions  are  absent,  the  ranking  described  in  3.6.2  will  be 
attempted. 

If  significant  interactions  are  detected,  one  of  the  two  ranking  schemes  de¬ 
scribed  in  3.6.3  and  3.6.4  will  be  attempted. 

A  final  ranking  will  then  be  done  as  described  in  3. 6.  6. 

The  total  experiment  is  a  mixed  model  of  fixed  treatments  consisting  of  three 
traffic  levels  and  four  modes  together  with  random  blocks  consisting  of  the  four  con¬ 
troller  teams.  *  Another  view  of  the  model  is  that  we  make  four  random  observa¬ 
tions  (controller  team)  on  a  12-component  vector  consisting  of  the  four  methods  and 
the  three  traffic  levels.  Ranking  will  be  performed  using  the  system  performance 
criteria  discussed  earlier. 

3.6.1  Operational- Procedural  Combination  Interactions  with  Traffic  Level 

The  multiple  comparison  of  means  to  be  described  In  3. 6.  2  assumes  no  inter¬ 
actions  between  traffic  level  and  the  operational-procedural  combinations.  This 
section  discusses  a  statistical  technique  of  testing  that  assumption.  If  significant 
interactions  are  detected,  one  of  the  two  ranking  methods  described  in  3.6.3  and 
3.6.4  can  be  used. 

It  is  desired  to  test  the  multiple  hypothesis  of  no  traffic  level  interactions  with 
the  four  procedural  modes: 

Ho:  ^111  ”  M1I2  ”  M12l  “  P122  ^211  ”  M2L2  M221  ”  M222 

P112  "  M113  '  M122  “  ^123  =  M212  "  M213  "  M222  ”  ^223 

where  Is  mean  performance  for  the  ij  operational-procedural  combination  at 
traffic  level  k. 

The  procedure  followed  is  outlined  In  Morrison  (13671,  pp.  186-196  for  profile 
analysis  of  q  responses  grouped  according  to  g  treatments.  In  our  case  there  are  3 
responses  corresponding  to  the  three  traffic  levels  and  4  treatments  corresponding 
to  the  4  operational-procedural  modes.  For  each  traffic  level  and  mode  there  are 
4  observations,  one  from  each  team. 

Let  be  ihe  system  effectiveness  using  the  it  operations  I -procedural  com¬ 

bination  at  traffic  level  k  with  team  l. 


‘This  is  discussed  more  fully  in  reference  6  pages  153-196.  Tables  arc  found  in 
reference  6  pages  312-319. 
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Form  the  differences  between  adjacent  traffic  levels 
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Compute  the  following  sums 
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sum  over  all  teams  on  adjacent  traffic  level  differences 
with  the  ij  operational-procedural  combination 


and 


2  2 

Gjj  -  £  E  T  ^  -  grand  total  of  all  differences  between  adjacent  traffic 
j-1  1  1  levels  summed  over  ij  operational-procedural  combina¬ 
tions 


2  2 


Hr.  1  *  4T,„TU.'w°rV‘ 


j  1  i  1 


2  2  4 

E  E  E  E  C  4  C 
rS  11  j  =  l  11  llri 


2  2 


'Msi  '  ^  4  Tijr  V  r  '  lt  2i  8  =  11  2 


It  can  be  shown  that  if  A  is  the  maximum  root  of 
m 


'Enhi  •  '  *  <*E12H12  -  rn«22  -  "nE22»  *  H!.H22  •  "L  *  »■ 


thenTTx —  f°ll°W6  the  Heck  maximum  characteristic  root  distribution*  with  param- 
m 

ctcrs  2,  0,  4.  5.  For  a  level  of  significance  a,  the  hypothesis  of  no  traffic  mode 
interaction  is  accepted  if 

X 

— j2-  si  «*)  2,  0,  4.5 
m 

where  >.  (on  is  obtained  from  tables  of  the  Heck  distribution*. 

Fora  .01  X  (or >  -  .775 


‘This  is  discussed  more  fully  In  reference  0  pages  159-196.  Tables  are  found  in 
reference  G  pages  312-319. 
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3.6.2  Multiple  Comparisons  of  Means  among  the  Operational  and  Procedural 
Methods 


If  It  is  found  that  no  significant  interactions  exist  between  the  traffic  levels  and 
the  operational-procedural  methods,  foe  statistical  tests  given  In  this  section  can  be 
used. 

Table  3  shows  a  portion  of  the  total  experiment  represented  by  Table  1.  The 
entries  for  the  other  two  traffic  levels  differ  only  In  the  subscript  k  representing 
traffic  level. 

A  2  x  2  table  of  averages  of  the  four  combinations  of  the  operational  and 
procedural  methods  for  foe  aggregate  data  from  foe  three  traffic  levels  can  be 
formed: 

Procedural  Mode 


i 

1 

M 

I 

■a 


P1 

P2 

Operational 

°1 

xir- 

X12‘  * 

Mode 

°2 

*21  * 

X22'  * 

In  the  notation  of  Table 

3 

E  * 
k=l  £1 


Ukl 


=  12  (S111  +  S112  * 


1  3  4 

X22' '  =  12  1  L  X22ki 

k  i  £-1 


12  <S221  '  S222  '  S223)- 
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Table  3.  DETAILS  OF  EXPERIMENTAL  DESIGN  FOR  TRAFFIC  LEVEL  1 


Methods  of  Operating  SAFOC 

O.P, 

o,p„ 

0„P 

0  P 

1  1 

1  2 

2  1 

2  2 

Xllll 

xmi 

‘Sill 

X2211 

XU12 

X1212 

X2112 

X2212 

V 

1113 

X1213 

X2213 

xni4 

X1214 

X2114 

X2214 

(Repeat  this 
for  each 
traffic  level) 


S 


111 


121 


211 


S 


221 


where  X 


Ilk/ 


X12k£ 
X21k£ 
X22k  l 


denotes  the  O^P^  effectiveness  measure 
denotes  the  O^P2  Effectiveness  measure 
denotes  the  effectiveness  measure 

denotes  the  effectiveness  measure 


Each  column  is  summed  over  all  four  teams  for  each  effectiveness  measure 


where  S 


111 


7  ,  X11U  ’  S12i 

l-  1 


4 


etc. 


The  practice  of  summing  -  jregatlng)  over  the  three  traffic  levels  assumes 
that  the  interaction  of  Traff.c  by  Operational,  Traffic  by  Procedural,  and 
other  similar  interaction  effects  are  not  statistically  significant. 

Assume  that  the  vector  of  means 


x  -  (xu..,  x12...  x21..,  x22>.) 

is  an  observation  on  a  four-variate  multinormal  random  variable  with  mean 
vector 

•*  Ml2’',  ^21”*  /i22”) 

and  symmetric  positive  definite  covariance  matrix 
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’*11 

*12 

°13 

*u 

1  T  1 

XT  £  *=  -r 

5 12 

°22 

*23 

*24 

N  4 

* 13 

*23 

®33 

*34 

*14 

*24 

°34 

*44 

m 

N  =  4  is  the  number  of  controller  teams,  while  the  elements  0  .  are  the 
variances  and  covariances  of  the  sums  *J 

*ljU  +  XiJ2£  *  Xij3£ 

of  Team  Z  summed  across  the  three  traffic  levels.  It  is  assumed  that 
these  variances  and  covariances  are  the  same  for  each  team 

From  the  elements  of  we  can  form  (|)  =  6  distinct  mean  differences. 
These  differences  and  their  estimates  are  shown  in  Tabic  4. 

Table  4.  DISTINCT  MEAN  DIFFERENCES  AND  THEIR  SAMPLE  ESTIMATES 


ftapulatlon  Mean  Sample  Estimate 


V 

"ll" 

dr 

*11" 

-*12** 

V 

;  Mir* 

•"21" 

V 

*11" 

*21" 

V 

■"n" 

-  f22-- 

V 

*11*  * 

‘  *22' * 

V 

M12" 

-F21.. 

V 

*12" 

*21" 

V 

^12*  * 

-  F22-- 

d. 

;> 

X12" 

*22'  ’ 

V 

^21" 

-  *22-  ■ 

V 

*2l" 

*22’  * 

In  order  to  test  for  homogeneity  of  means,  test  simultaneously  the  six 
null  hypotheses: 

H  :  6,  o,  I  1 . (i, 

o  I 

of  no  treatment  combination  mean  difference  at  some  overall  level  a  (e.g. , 
0.05  or  0.01).  Individual  tests  will  not  satisfy  the  maintenance  of  a  con¬ 
trolled  significance  level  a  liecause  in  making  such  multiple  tests,  the 
effective  overall  significance  level  is  raised.  Furthermore,  the  possible 
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dependencies  of  the  test  statistics  also  obscure  the  true  error  rate.  These 
difficulties  for  a  set  of  hypotheses  such  as  the  six  suggested  by  the  6(  can 
be  overcome  by  constructing  confidence  intervals  from  a  theorem  in 
probability  known  as  Bonferronl's  inequalities  (Feller  1957,  p.  100).  Details 
of  its  application  to  simultaneous  statistical  inference  have  been  described 
by  Miller  (1966).  Essentially,  for  a  set  of  confidence  Intervals  on  m  means 
with  confidence  coefficient  at  least  1  -  or,  one  uses  the  usual  Intervals  with 
the  (hstribution  critical  value  t0/2.n  replaced  by  t^/jm-n.  In  our  case  we 
would  replace  a  =  0. 05  by  0.00833  if  we  wished  a  probability  of  covering 
in  excess  of  0. 95.  The  general  Bonferronl  interval  is 

\  ~  *  6h  '  *  t0t/12;nSdh‘ 


where 


;  h  *  1,  2. . ,  6 


d^  is  the  hth  mean  difference  between  operational  and  procedural  combina¬ 
tions  for  team  £.  h  =  1 . 6 


dli  ”  Xlllf  ‘  X121£  +  X112i  “  X122£  +  Xil3t  ‘  X123£ 


likewise  for  d„, ,  d.  .  ....  d„  ,  in  the  general  Bonferronl  interval. 
2  *  o  £  b  l 


Here 

n  -  degrees  of  freedom  =  number  of  teams  (4)  minus  1  =  3 


u 

i 

3 


and 


=  standard  error 


h  =  1,  2.. 6 


i.e.  the  standard  error  cf  the  hth  operational  procedural  combination 


1 

/N 


(times  the  standard  deviation  of  the  differences  of  the  team 
scores  for  the  hth  combination). 


Ilf 


ilttB UlmMiJk**-*-**"  twWattt.1^  .  .  *•  H.U** 


For  N 


4 


where  d j,/  «  hth  mean  difference  between  operational  and  procedural  corn- 
bln&Uona  for  team  i ;  h  -  1,  2,  ...  6. 


For  n  =  3  degrees  of  freedom  we  have  the  following  critical  values  (where 
n  =  number  of  teams)  (by  linear  interpolation  in  tables  of  the  t  distribution): 


a  la/12;3  4a/2;3 

0.05  6.23  3.18 

0.01  12.  (approximate)  5.84 


For  the  ranking  of  the  four  combination  means,  proceed  in  this  manner: 

(1)  Arrange  the  four  combination  treatment  means  in  ascending  order: 


Xi](min)  *  XiJ(mnx) 

or,  introducing  a  new  subscript  notation  (il  for  the  Jth  smallest 
mean, 


X  5  X  i  X  £  X 

ll]  I2l  (3)  (4] 


(2)  Compute  the  standard  deviations  of  the  six  pairs  of  differences. 


(3)  Compute  the  six  confidence  intervals  for  the  distinct  population 
differences  flj,  ...  flG.  If  the  Jth  confidence  interval  doeo  not 
contain  zero,  reject  the  null  hypothesis 


H  :  6=0  for  all  J,  i  -  1,  ....  6 
o  j 


and  conclude  that  the  means  of  the  two  combinations  in 
different. 


are 
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(4)  Draw  horizontal  tinea  under  each  pair  of  means  xm  j,  xjjj  whose  differences 
are  oot  significantly  different.  If  a  mean  Is  statistically  different  from  each 
of  the  other  three,  denote  this  by  a  single  horlaontal  line  under  It.  A  per¬ 
fectly  unique  ranking  would  have  no  joining  lines.  If  the  smallest  two  means 
were  not  significantly  different,  yet  the  remaining  means  were  significantly 
different  from  each  other  and  the  first  two  means,  the  configuration  would 
be 

xmxt2)xi3)xt4r 

It  should  be  emphasized  that  the  multiple  tests  may  not  lead  to  a  distinct 
ranking.  For  example  the  following  ranking  is  not  distinct. 

XU1  X|21  Xf31X|4) 


3.6.3  Multiple  Comparisons  within  a  Traffic  Level 

The  method  of  paragraph  3.6. 1  can  be  used  to  determine  if  significant 
Operational-Procedural-Traffic  level  interactions  exist.  If  these  interactions  are 
significant,  rankings  of  operational  procedural  combinations  must  be  made  within  a 
traffic  level.  The  method  1b  the  same  as  that  used  in  3. 6.  2  to  rank  the  four  Oj 


.’58 


mr* . ~i*m 


combination  means.  The  degrees  of  freedom  and  critical  values  remain  unchanged, 
but  the  various  differences  and  means  must  be  appropriately  rodeflnod.  For  example, 
the  combination  means  arc: 


xiik*  *4  .L,  xnki!  k  * l'  2>  3 


-  i 

x„„,  .  =  7  I  x„„,  k  -  1,  2,  3 
22k  4  j  22ki  ’  ’ 

To  test  the  multiple  hypothesis  of  no  difference  between  the  population  moans  of  the 
combinations  OjPj  and  O^P0  at  Traffic  level  1,  we  calculate  the  four  differences 

eill  ‘  Xlill  "  X1211 


C1U  "  X1114  ’  X1214 


their  mean 


en  *  xnr  *  X121* 

and  the  standard  error  of  that  mean  as  in  the  previous  section  with  N  1  and 


s- 

C, 


hk 


■*V>[ ] h  '■*■■■■* 


k  --  1.  2.  3 


The  test  and  the  subsequent  ranking  of  the  four  combinations  is  as  described  before. 
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3, 6.4  Multiple  Comparisons  and  Ranking  of  the  Twelve  Traffic- Ope  rational” 
Procedural  Combinations 

In  order  to  eliminate  interaction  effects,  it  la  theoretically  possible  to  rank 
the  twelve  means  of  the  complete  experimental  design,  but  since  this  leads  to  (i*>  2 
66  pairs,  the  effect  on  the  t_  critical  value  is  such  that  the  resulting  tests  woulabe 
inordinately  conservative.  A  more  reasonable  approach  might  consist  of  a  ranking 
of  the  Traffic  levels,  followed  by  rankings  within  each  Traffic  level. 


In  the  same  notation,  the  three  Traffic  level  means  are 


x..,.  =  —  Z  R 


1  16 


i=l 


U 


x. . 


2'  16 


4 

Z  R 
M 


2i 


X*  ‘  .V  16 


=  Z  R 


A=1 


•ere  Rj^j  =  sums  of  observations  on  operational  and  procedural  combinations  for 
ihe  kth  traffic  level  and  Jtth  team. 


2  2 

*kX  =  ^  XiJki  =  x. .  kl 

These  mean';  can  be  ranked  by  computing  the  Bonferroni  simultaneous  confidence 
intervals  1  on  their  mean  differences 


U1  “x..l. 


U2  =  X..2. 


U3  ’X..l. 


. .  2. 


. .  3, 


-  x 


..3. 


Here, 


[f  f  "3.]  '11  \  ,RU-RW’ 
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U  ws  denote  the  successive  level  differences  for  the  teamB  by  the  ujj  symbol  used  in 
the  matrix  of  level  differences  and  introduce  the  new  differences 

U31  “  R11  ~  R3jt* 1  "  l*  ‘  *  •  *  4‘ 
then  letting 


U3  '  16 


4 

L 

1=1 


u 


31 


4 

L 

1=1 


10 


apply: 


For 

N  =  4,  let  x  =  -7= 
4 


The  three  standard  errors  of  the  level  mean  differences  can  be  written  as 


Since  there  are  three  mean  differences  the  critical  value  is  t  .  : 

u  /b;j 

_ *0/6:3 

0.06  4.87  (linear  interpolation) 

0.01  8.G  (linear  extrapolation) 

For  example,  the  95 %  confidence  interval  for  the  population  mean  Pj  of  Uj  is 

u.  -  4.87  8-  S  /i  *  u ,  ♦  4.87  s~ 

1  u.  H1  1  u. 
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If  such  an  interval  contains  zero  we  would  accept  the  hypothesis 


Hq:  ^  =  0  for  each  i;  i  1,  2,  3 

and  conclude  that  the  Traffic  level  grand  means  are  not  significantly  different.  We 
would  rank  the  Ti'affic  level  means  in  the  same  manner  as  in  paragraph  b.G.  2.  A 
distinct  rank  is  assigned  to  those  means  that  are  statistically  significantly  different 
from  the  other  two  means. 

3.G.5  Justification  of  the  Bonferroni  Inequalities  Approach 

The  usual  multiple  comparison  method  for  repeated  measurements  data  is  one 
commonly  due  to  Scheffe^  (1959).  When  the  number  of  multiple  tests  is  large,  the 
Scheffe^  confidence  intervals  will  be  shorter  on  the  average  than  those  given  by 
Bonferroni's  inequalities,  provided  the  sample  size  is  moderate  to  large.  Since 
our  sample  of  N  4  teams  is  exceptionally  small,  and  we  are  only  interested  in  a 
few  comparisons  of  the  means,  the  Bonferroni  approach  is  decidedly  preferable. 

3.  G.  G  Final  Ranking 

The  system  effectiveness  measures,  listed  in  paragraph  2,  are  the  X.  .  . 
parameters  used  in  the  preceding  discussion.  Operational  procedural  combina-^ 
tion  rankings  will  be  performed  for  each  effectiveness  measure  separately.  The 
final  ranking  will  be  made  by  summing  the  effectiveness  measure  ranks  for  each 
operational-procedural  combination.  The  combinations  will  then  be  ranked  ac¬ 
cording  to  lowest  rank  sum  to  determine  the  best  opcrationnl-procevlural  combina¬ 
tion. 
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3.  7  MATHEMATICAL  MODEL  AND  CONTROLLER  TEAM  PERFORMANCE 

3.7.1  Objective 

The  objective  of  this  test  is  to  determine  when  the  SAFOC  controllers  have 
reached  an  asymptotic  limit  in  learning  the  SAFOC  controls.  This  will  be  performed 
by  measuring  how  well  each  of  the  four  teams  performs.  This  test  will  be  repeated 
to  determine  the  learning  curve  of  operators  performing  similar  tasks.  The  effec¬ 
tiveness  measures  listed  in  paragraph  3.5. 2  will  be  used  for  controller  performance 
evaluation. 

It  is  possible  to  rank  the  controllers  on  some  measurable  characteristics.  For 
this  ranking  the  C  teams  in  R  replications  using  different  but  equivalent  scenarios  will 
be  used.,  in  essence,  random  trials  of  the  controllers'  ability  to  handle  repeated 
scenarios  will  be  performed  to  determine  whether  the  controllers  are  significantly 
different  in  their  abilities.  If  they  are  different,  they  will  be  ranked  in  order  of  their 
abilities  to  determine  those  needing  additional  training. 

If  new  personnel  must  be  introduced  during  the  tests,  attempts  must  be  made  to 
train  them  to  the  approximate  proficiency  levels  of  the  experienced  controllers  for 
the  results  to  have  significance. 

3.7.2  Statistical  Factorial  Design  Mathematical  Model 

Table  5  demonstrates  the  symbology  of  the  effects  (effectiveness  measures*  re¬ 
sulting  from  replications  of  different  scenarios  by  each  controller  and  indicates  the 
sums  to  be  performed.  Table  6  indicates  the  usual  analysis  of  variance  for  a  two-way 
classification  based  on  a  mixed-model  of  fixed  teams  and  random  samples  from  the 
hypothetical  population  of  replication  observations. 

When  the  seenarios  are  run,  the  data  (such  as  the  workload  times  of  each  con¬ 
troller)  consisting  of  the  X(j's  filled  in  on  Table  5  is  operated  on  by  performing  the 
column  and  row  sums,  followed  by  the  operations  called  out  in  Table  G.  Then  the 
sums  of  squares  and  mean  squares  called  for  in  Table  G  are  computed.  The  F  ratio 
is  computed: 


(ft  -  1) 


where  S.>.  S.{  and  R  are  defined  in  Table  G.  Reject  the  null  hyiiothesis  of 
no  team  difference  if  fit  exceeds  the  a  |>oint  of  the  F  distribution  with  (C-li 
and  v It-li  (C-lt  degrees  of  freedom.  For  example,  if  a  equals  U.Oo,  C 
equals  four  and  It  equals  two.  then  the  critical  value  is: 


Replications 

1 

2 

B 

B 

c 

Row  Sun 

1 

X11 

X12 

1 

1 

X1C 

f  x,j  •  c\. 

2 

X21 

X22 

1 

1 

X2C 

C 

=  *21  ‘  CX2. 

* 

• 

• 

B 

B 

• 

• 

• 

• 

B 

1 

• 

R 

*ri 

XR2 

1 

1 

’Sic 

f  XK,  *  cSR. 

Column  Sum 

R 

rx  -Ex 

t  1  |  11 

R 

E  X  =  RX  „ 
t  12  .2 

l 

“  c  fV 

R  C 

E  Z  X  RCX. . 

1  J  J 

C  Number  of  teams 
R  ■  Number  of  replications 


Table  6.  Analysis  of  Variance  for  a  Two-Way  Classification 

DEGREES  OF 

SOURCE  SUMS  OF  SQUARES  FREEDOM  MEAN  SQUARE 


Replications 

R  -  -  2 
s  ec:  -  X  ) 

i 

R-l 

Sj/fR-l) 

Teams 

C  _  _  2 
s„  E  (x  -  x  r 

C  j  •  J  •  * 

C-l 

S2/(C-1) 

Error 

, 

V“«VXI.- V*..' 

(R-D  (C-l) 

S3/tR-l)  (C-l) 

TOTALS 

RC  -  2 
s  ZZ  (X  -  x  f 

i  )  1J 

RC-1 

— 

^  *0,  06,  3,  3  “  9‘  28 

If  the  computed  value  of  F  was  larger  than  9. 28  one  could  say  that  the  con¬ 
trollers  performed  their  tasks  with  significantly  different  capabilities  and 
that  similar  tests  would  repeat  this  conclusion  with  a  6%  risk  of  being  in 
error. 

If  we  wish  to  rank  each  of  the  teams  with  respect  to  every  other  team, 
TXikey's  multiple  compatison  procedure  can  be  used.  This  consists  of 
taking  the  difference  between  every  pair  of  team  performance  means  com¬ 
puted  during  the  test: 

(3,  Bx  -  B2  -  Zj 

V  VZ2 


B1  -  B4  =  Z3 


B  -  B  s  2 
2  3  4 


B,  -  Bu  = 

3  4b 


where  B  is  the  mean  workload  performance  parameter  of  £th  team 
l 


1 


H 


V4£xu 

l 


We  then  compute: 


,4)  V/T 

where  S  -yj S  /  (C-l»  <  H- 1  > 

u  -  critical  value  for  the  Studentizcd  Kange. 

a 

The  value  of  is  extracted  from  the  indicated  table  at  some  value  of  o 
(say  0.05t  and  for  the  error  degrees  of  freedom  (3  in  Uie  example i.  If  (4) 
contains  zero  (that  is,  if  the  left  hand  term  is  negative  and  the  right  hand 
term  Is  positive)  then  the  difference  is  nut  significant.  This  is  per¬ 
formed  for  the  remaining  Z's  and  rankings  are  achieved  on  the  ba9is  of  the 
relative  values  of  (4>.  For  example, 
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(5)  Ba  Bt  B, 


3.7.3 

a. 


where  the  underlines  might  indicate  no  significance  between  differences  in 
B£  and  Bj  but  significant  differences  in  B2  and  Bj.  We  would  then  conclude 
that  Team  2  is  not  significantly  better  than  Team  1  (at  the  3%  level  of  sig¬ 
nificance)  but  that  the  difference  in  performance  between  Team  2  and  Team 
4  is  significant.  This  result  could  be  used  to  indicate  that  Teams  3  and  4 
needed  additional  training  to  bring  them  up  to  Team  2  and  Team  1. 

For  computational  purposes,  the  row  and  column  sums  will  be  summed  and 
then  squared,  rather  than  summing  the  squares  as  shown  by  the  operations 
in  Table  0: 
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R  C  „  R  C 
E  i£  X  )  -  <£  £  X  ) 

i-j _ : _ l_j _ L 

rc 


c  B  2  R  c 

I  <5  x  )  -<£  £  X  I 

8  - *  J- 

2  HC 


s  =  s  -  s  -  s 

3  4  1  2 


S,  *  RC 

♦4 


R  c  2  R  C 
£  £  X  -  (I  £  X  , 

LJ,  j  I  J _ _ 

RC 


Criteria 


INDEPENDENT  VARIABLES 

The  independent  variables  for  the  controller  team  comparison  test  arc: 
Traffic  Density  Level 

The  total  number  of  flights  simultaneously  in  the  SAFOC  control  area. 
Replications 

The  repetition  of  similar  (but  not  necessarily  identical)  scenarios. 


2fif> 


3.  Controllers 


Tests  will  be  performed  with  four  teams  of  two  controllers  each. 

b.  DEPENDENT  VARIABLES 

1.  Safety 

2.  Controller  Workload 

3.  Communications  Workload 

4.  Delays 

5.  Throughput 

6.  Capacity 

7.  Uncontrolled  Time 

c.  CONSTANTS 

The  same  map  will  be  used  by  all  teams  and  for  all  replications.  Altitudes 
and  other  flight  plan  data  will  be  held  constant.  Mix  of  aircraft  speeds,  air¬ 
craft  types,  aircraft  equipment  will  be  constant. 

The  same  number  of  scenario  replications  will  be  played  to  each  team.  Sim¬ 
ilar  scenarios  will  be  used. 

A  fixed  traffic  level  will  be  used  in  lilts  experiment. 


APPENDIX  A 

NOTATION  CONVENTIONS 


For  subscripted  variables  such  as  the  following  conventions  apply: 

If  a  subscripted  variable  la  summed  over  the  entire  range  of  a  subscript, 
the  result  is  denoted  by  the  subscripted  variable  with  a  dot  replacing  the  index 
over  which  the  sum  has  been  taken. 


For  example: 

R 


V 


P  M  N  R 

X. ...  =  L  L  L  I 

1=1  k=l  j=l  i=l 


If  a  variable  Is  to  be  averaged,  a  bar  appears  over  the  variable 
thus. 


Y  -  i.  V 

i.kl  N  l.kl 


or 


X. . . .  - 


i 


PMNR 


1  N 

N  f=1  V 


PMNR 


P 

L 


M 

L 


N 
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1=1  k=l  1=1 


P. 

L 

i=l 
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APPENDIX  B 

DEFINITION  OF  SYMBOLS 


Swabol 


1  • •  •  • 


C 


D 


12 


V 


®hki:  ®U1 . ®634 


F 

h 


Definition 

Performance  Parameters  for  four  teams 


1  R 

1  R  1=1 


Xu/=1,  2,  3,  4 


B i  -  mean  workload  performance  parameter 
of  ith  team 

Number  of  columns  (teams) 

1  4 

Mean  difference  =  -  £  u  , 

4  1=1  11 

Mean  differences  d^  =  x^  -x^ 

Note  that  f=1  “hi 

th 

lr—  mean  difference  between  operational  and 
procedural  combinations  for  team 
2  .  h  =  l . ,6 

th 

h  difference  ahnong  the  four  Operational  and 
Procedural  combination  observations  for 
team  t  at  traffic  level  k: 

Mean  of  operational  and  procedural  differences 
averaged  over  the  teams. 

Ratio  uaed  in  statistical  teats 

Index  indicating  one  of  six  mean  differences 

Null  hypotheses 
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Definition 


Symbol 

1  =  1,  2  Operational  modes 
J  =  1,  2  Procedural  modes 
k  =  1,  2,  3  Traffic  levels 
l  -  1,  2,  3,  4  Teams 
m 


N 


Number  of  means  for  which  simultaneous 
confidence  intervals  are  to  be  computed 

Number  of  events  or  measures 

Degrees  of  freedom  (no.  of  teams  -  1) 

Operational  method  with  two  sectors 

Operational  method  with  one  sector 

Procedural  method-normal 


P 


2 


Procedural  method-passive 


°lP2-  °iPj 


a 

% 


Operational  procedural  combinations 


Critical  value  for  the  Studcntizcd  Range 


Row  sums  of  observations  on  the  Operational 
and  Procedural  mode  combinations  for  the 
kth  Traffic  level  and  ith  team. 


“kf 


2  2 
Z  Z 
J-l  i-1 


X 


ijk* 


Column  sums  of  the  basic  data  matrix 
(observations  under  Procedural  mode  i. 
Operational  mode  J,  and  Traffic  level  k 
summed  over  tne  four  teams). 


4 

V  "  f  x  Vf 


2  70 


Symbol 


Definition 


2 

a  - 
x 


Standard  Error  of  the  hth  Operational  Pro 
cedural  Combination  Mean  Difference: 


h  =  1, . 6 


Standard  error  of  the  hth  Operational  Pro¬ 
cedural  Combination  Mean  Difference  for 
Traffic  level  St: 


B~ 


Standard  error  of  the  gth  traffic  level  com¬ 
bination  mean  difference: 


g  -  1,  2,  3 

Standard  error  of  mean  of  x 


S  ...S,  Sums  of  Squares 

1  4 


u 


U 


Itoot- mean- square  error 

Statistic  used  for  testing  hypotheses 

Critical  value  tor  statistical  test 

100  q/o,c  critical  value  of  Student- Fisher 
distribution  with  J  degrees  of  freedom 

Symbol  used  to  dcnoic  the  successive  level 
differences 
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* 

r 

r 


r 


t 


Vi 

*ljk£ 


U 


Vrxm 


X 


X 


"k£ 


X. 


•  V 


XRl’  XR2“XRC 


X 


R 


X 


11 


Average  of  level  differences  (u^  averaged  over  )) 
Is  some  effectiveness  measure  (e.  g. .  workload) 


Observed  value  of  variable  X  for  Operation  mode  1. 
Procedural  mode  j,  Traffic  levek  k,  and  Team  t. 

i  »  1,  2  j  *  1.  2  k  »  l,  2,  3  /  •  1.  3,  4 

Combination  treatment  mean  for  Ith  operational 
and  jth  procedural  combination. 

Xll|  Xf2J*  *  *  X(m) 

Ranks  -  ordered  set  of  means 

Is  an  observation  on  a  four' variate  multinormal 
random  variable  with  mean  vector 

rlV"  V-*  V’*  v-» 

the  4x3  matrix  of  team  sums  for  each  traffic  ievel 


2 

Z 

1=1 


2 

Z 

1=1 


x.., 

ijk£ 


Traffic  level  mean 


K,  . 

‘i 


The  data  (such  as  the  workload  times  of  each 
controller)  consisting  of  the  X^j ' s 

Mean  performance  measure  of  Rth  replication 
(averaged  over  C  teams) 

Replication  one  for  team  1  of  performance  meas¬ 
ure  for  controller  team  evaluation 
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Definition 


Symbol 


Z 


6 


■*  P 


22‘ ' 


Differences  in  team  performance  means 

Operational- Procedural  Combination  Differ¬ 
ences  (population  mean) 


6i  “E(di)  =  Mn..  -*12.. 


«£sE(d2)  =  #,n..  ^21.. 


fl3  =  E  (CV  =  MU. .  "*22. , 


fi4  =  E  <d4)  =  M12. .  ”**21. . 


*5  =  E  (d5)  =  ^12..  '**22.. 


fi6  '  E  (d6>  .  "M22.. 


Mean  four-variate  multinormal  random  vector 
=  p22--) 

Components  of  p 
Population  mean 

Controlled  significance  level 
ij  element  of  covariance  matrix 
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PART  II 


NEW  ANALYSES  A>.D  METHODS  LEADING  TO  IMPROVED 
TARGET  ACQUISITION  REQUIREMENTS  INVOLVING  SYSTEMS, 

GEODETIC  AND  RE-ENTRY  ERRORS ,  AND  INCREASED 
WEAPONS  EFFECTIVENESS  FOR  CONVENTIONAL  WEAPONS 

Hans  Baussus-von  Lueczov 
U.  S,  Army  Engineer  Topographic  Laboratories 
Fort  Belvoir,  Virginia 

SUMMARY.  The  paper  represent*  a  supplemental  analysis  for  height  bursts 
as  well  as  vertical  target  location  errors,  considering  flat  and  contoured 
terrain,  and  thus  completes  the  development  of  optimal  methods  for  weapons 
research  and  development  and  a  broad  spectrum  of  requirement  analyses. 


FOR WORD.  This  study,  appearing  as  Part  II  under  the  same  title  as  a 
prior  one  presented  at  the  Fifteenth  Conference  on  the  Design  of  Experiments 
in  Army  Research,  Development  and  Testing,  in  October  1969,  completes  the 
rigorous  treatment  through  extension  to  three  geometrical  dimensions  of  the 
fragment  damage  problem  pertaining  to  tube  and  missile  artillery.  It  may  be 
followed  by  a  similar  analysis  for  conventional  cratering  and  for  nuclear 
weapons.  As  emphasised  before,  implementation  of  the  methods  and  concepts 
developed  would  undoubtedly  lead  to  a  significant  increase  of  Army  weapons 
effectiveness.  In  addition,  the  new  methods  are  expected  to  have  some  rami¬ 
fications  pertaining  to  a  variety  of  research  and  development  and  combat 
development  activities.  The  technical  responsibility  for  this  study  is 
exclusively  that  of  the  author,  who  appreciates  the  U,  S.  Army  Engineer 
Topographic  Laboratories'  continued  interest  in  this  kind  of  effort. 
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NEW  ANALYSES  AND  METHOD.'  LK  TO 

IMPROVED  TARGET  U  OL  ISITION  REQUIREMENTS 
INVOLVING  SYSTEMS,  GEODETIC  AND  RE-ENTRY 
ERRORS.  AND  INCREASED  WEAPONS  LEE  EC  1 1  YEN  EE’S 
FOil  CO N  VENTIONAI.  WEAPONS 

l.  Introduction.  The  author  ha.-  shown  in  a  prior  p;tp<T.  referenced  a*  U70-2.* 
that  tin-  artillery  problem  in  i»m,  geometrical  dimensions  ran  he  formulated  in  terms  of 
four-dimcn-iorsal  integrals  ami  that  optimization  for  multiple  volleys  is  pos-ihle.  In  thi- 
way.  two-dinirn-ionul  weapons  di-per-ioji.  meteorological  irror?  (v.imh.  a‘r  il<,n*ili'>). 
target  location  errors  (including  identification.  intelligence.  map).  -i/»-.  -hap*-,  ami  tar¬ 
get  distribution  (drii-ilv)  ean  l><*  considered  sitnitllaneoti-ly .  i.e..  an  integrated  effeet  in 
term-  of  expected  ea-nallv  re-nit-  as  a  function  of  multiple  volleys. 

For  three  geometrical  dimen-iotis.  il  is  necessary  to  utilize  the  Intr-t  height  di-tri- 
billion  [larameters  for  proximity  ami  time  fuzes  (for  a  constant  range  and  impaet  anile) 
and  the  vertiea!  target-location  error  (mean  height  error  of  target  distribution)  and  their 
associated  statistical  distribution-.  assumed  to  he  normal. 

Fortunately,  the  former  analytical  approach  can  he  systematically  extended  to  six 
dimensions,  and  respective  optimization  conditions  can  he  written. 

In  order  to  simplify  the  prohh  m  and  permit  evaluation  cf  improvement*  as  well  a* 
sensitivity  analv  ses.  the  target  mav  lie  a— limed  to  he  circular  and  homogrivniis  am!  rrf- 
crenccd  to  a  center  of  gravity  under  consideration  of  target-density  indices. 

If  the  target  mean  height  variance  is  small,  the  optimization  pattern  for  n  volleys 
to  he  invariant  with  respect  to  a  rotation  is  practically  two-dimensional.  In  case  of 
greater  mean  height  variances,  competitive  three-dimensional  optimization  configura¬ 
tions  arise,  although  they  are  generally  restricted  to  the  center  of  gravity.  e.g..  three 
competitive  aiming  heights.  Comparison  of  numerical  optimization  results  easily  j*-t- 
mils  establishment  of  the  correct  ranging  order. 

Inclusion  of  the  shape  of  tin-  terrain  doe*  not  provide  any  difficulties  and  is  pro- 
vidrd  for  in  the  general  formula-,  although,  in  practical  situation-,  it  can  hardh  he  con¬ 
sidered  and  its  effect  -hould  rather  lie  added  to  tic-  target  mean  height  varianc' . 


1.  Xl«l  V.  ali-  -  jmt  Lr-uliii;  t->  Inipnsnl  R  Tir.  -v  nt*  Tnvo’ain;  S.-Iri-..  V...!efic 

and  Re*  ntrv  K.r?<»r>.  .mi!  In.  rrj-.-.l  Wrj-mu-  EnV,  :ii,  n.v  f<>-  I'omrnUorol  W^jp.»n- (Tart  !)i  AttO-D  R,|*»rl 
of  the  Fitter  >iih  e.v.r  ■>«  t!ir  IV->  ol  t. in  Vrmv  li.  Por>,>;,m--nt  amt  Tr-tm-. 

Tin-  p.i|*r  v-  st-»  jtaiijMe  a-  Is  A  I.  tl.  Rr-ejrvli  Note  No.  A».  Af»  roi‘l’1. 

jReproduced  from  3^ 
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II  is  I’li'ar  that  tin:  inclusion  of  two  additional  dispersion.-  result.-  in  lower  iiv-m 
casualties  or  requires  more  vt >1  !*.-\ re.-pertivcly.  For  this  rea-on.  the  reduction  of  tar¬ 
get  identification  and  intelligence  errors  !»y  direct  target  acquisition,  including  relative- 
ly  accurate  location  of  controlled  weapon  delivery  system-  and  forward  oh.-erver* and 
the  utilization  of  the  contemplated  Position  and  Azimuth  IVlcnmuiug  .Sy  stem  (PA1>>) 
and  the  Long  Range  Positioning  IVtermimng  System  (LRI'DS).  is  of  importance.  For 
louder  ranges  and  under  a  variety  of  conditions.  these  -v-terns  might  not  he  avaitahle 
on  a  timely  basis  or  rr.ighl  not  function  properly,  -o  that  optimal  methods  for  multiple 
volleys  to  lie  incorporated  in  TACFHlF.  are  indispensable.  It  follows  also  from  margin¬ 
al  utility  considerations,  as  outlined  in  1*70-2.  that  four  or  more  volleys  to  he  fired  in 
optimal  patterns  arc  advantageous  for  shorter  ranges,  where  effectiveness  is  relatively 
high.  In  this  way,  the  problem  of  map  accuracy  per  sc.  at  least  with  respect  to  cta-s  1 
and  2  maps,  can  he  completely  resolved.  Fortunately,  target  mean  height  inaccuracies, 
often  envisioned  at  longer  ranges  despite  relatively  precise  determination  of  horizon fn! 
coordinates,  can  also  he  overcome  by  optimal  volley  patterns. 

Based  upon  strategic  consideration,  it  appears  that  a  combination  of  improved 
positioning/targct-ai'ijuisition  deviees  or  systems  and  optimal  multiple  volley  fire  based 
on  a  set  of  input  descriptors  is  particularly  cost  effective  in  ease  of  a  favorable  force 
ratio  and  cost  effective  as  well  as  decisive  in  ea*r  of  a  force  ratio  of  about  one.  Such  a 
combination  can,  of  course,  not  compensate  for  a  highly  unfavorable  force  iaiio.  Fur¬ 
thermore.  such  a  combination  is  expected  to  he  most  effective  in  small-  to  medium- 
intensity  conflict. 

Since  optimal  multiple  volleys  arc  highly  effective- when  there  is  ample  ammuni¬ 
tion  supply  ami  strong  compensation  for  errors  of  all  kind* -and  cost  effective  when 
ammunition  is  limited,  preparations  for  their  incorporation  in  TACFIRF.  should  start 
as  soon  as  possible.  The  great  advantage  of  optimal  multiple  volleys  is  their  flexibility, 
pertaining  to  the  combat  situation,  including  availability  of  position  and  target-aequbi- 
lion  equipment  and.  last  but  not  least,  they  eliminate  the  need  for  stringent,  rigid.  ami 
often  inaehievable  design  specifications  for  new  equipment. 

2.  Formal  Extension  to  Include  Burst  Height  and  Vertical  Target  i.«»cation  Var- 

O  O 

■ability.  The  definitions  am!  nomenclature  of  U70-2  are  maintained  and  extended. 


As  to  weapon  distribution  parameters,  we  add  tin-  vertical  standard  deviation  Sf. 
In  case  of  a  proxhnity  fuze,  wc  have  then  the  eomph  te  range  variance 


o  o  o<> 

S“  =  S  “  +  coin-  P  S 
r,p  r  K  z 
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\vi 111  ,3  a-  I I •  \  impai  I  angle  (<  !’.  1  iu-  1  m  !’* 


Tile  lal'Ta!  variance  remain-  h:mg:i!. 
If  a  time  fn/.e  is  applied.  we  obtain 

3“  .r  =  v7.  In"  l3  var  T 

■/.,  r  h 


(!■') 


o  •>  .» 

S%.  =  S"  +  v"  coin"  £  var  T 
r.1  r  K 


where  \j.-  is  the  terminal  projertile  velocity,  var  T  the  time  fn/e  variatem.  and  .''r  |  the 
composite  ratine  variance.  Il  should  he  einpfia.-i/ed  again  that  all  equation-  refer  t* » e. 
constant  range. 


The  vertical  target-location  di-tribnlion  (with  refu-vee  t<»  l!»<-  mean  height  j  i- 


The  aiming  point  for  the  fir-t  volley  now  ha-  the  eoordinates  jj.  h|.  dj  (the  letter 
c  has  already  heen  n-e<l  as  a  configuration  paramel-r). 


After  these  preparations,  am!  denoting  the  weapon  ili-lrihntion  parameters  for 
simplicity  hv  .Sr,  $.f  .  eqs.  (IS)  —  (t<)  of  II 70-2  van  he  generalized  a-  follow#; 

For  the  first  vollev.  wr  have  fo:ir  !>ur-l  di.-lributions  designated  hv 

•  O  • 


X ,  (\.v :  a  t  +  £  .  h )  s-  | :  «r  <t)  d\dy  5  (z:  d , ;  S,)  .1/.. 
X.>(\.y:a|  -•£  I>  j  +  £  :  Sf.  S^)«1vly  5  (/.:ilj:  «!/.  ete. 


(») 


where  5  represent.-  the  snpplementa!  normal  height  of  -hurst  di-trihution. 

For  snffieienlK  small  volume  element-  J  \A\  A/..  v\e  arrive  then  ir.  integral  form  at 
an  intermediate  average  probability  of  hitting Tj  at  lea-t  oe.ee. 


•v 
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-OOO1 


(') 


and,  since  Oj  ohcy a  di'trihulion  law .  at 


■OOOB  -  OO 


■filing 

—  oo  o  —  co  1 


5  *  1*1  (x.y.z:  fc.  TJ.  TT|.  fj)  . 


f(t .  n)  !•>(£)  dxdy  d/d$d;;d? 


(6> 


For  k  *  ami ;» t»!u.sl  l.:r;;el>.  wr  obtain  the  total  expected  «vna1S\  I'  -.'U 

r*k  n  N  i.00  0" 


niirlZ  T.  ||  j  |jj  2Z  \ 

f:l  Ji  !  .  «  i)  .;g  1 


a  ./:  f.rj.v:  J  ,r?  a  ) 
s  ‘  4  ‘fi  ,u 


Optimal  aimiii’i  coordinate  triple  seN  h.oc.  of  ••••tirM*.  ti>  be  «>i>taiTt***I  by 
methods  and  •>|*|ir*»priat«*  rankinj:. 


For  a  circular  and  homogeneous  (uniform)  target  distribution.  tin  condition'  (7) 
reduce  to  fewer  equation*.  i.c..  the  re*nci  !?\e  aimhi^  pattern.  conri-tiu;:  of  a  --.  l  of  k 
origins  Oo,  would  he  imari.ml  under  a  rotation  ahotit  Oj. 

After  the  first  %o!!ey,  winch  ha*  in  many  e.m  •  a  surprise  effect,  a  d.  uoa'.itinn 
with  rc*peet  to  I’j  ran  he  expected  which  can  he  exon — •  <!  a*  a  trau-iiion  from  •ytoun*! 
to  air  hurst’  and  hy  an  empirical  reduction  factor  li  <  [ .(If).  Taking  llii-  into  cn»*ider 
ation,  a  more  General  re-ull  correspondin':  to  cj.  (7)  read'  a* 


2.  Cf.  I  <w>i„(>tf  *  t»r  KTrt-J, 
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f»n  )-  \  ♦  COOT  SO 

P  ■'  rr  free 


P«  »  7  -  I  w  9  *  0 

■I  -  CO  O  - 


f(£.rj)  l'.j(f)  dxdydzdSdrjd* 


Til  |I*-):Y  *00  00  *  OO  . 

T-  ^  -  .>  ...  .  »,.r| 
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+  H  II  R  }|  j  |j|^Xv  5  V2)  ‘ r|C(x-.v  /;i^->: 

T=2  -mo  -oo 


•  rd.n)f2(f)  .lx.!v, |Zflj,!^lf  (9) 

It  should  lie  noted  1 1 m l  .lie  coupling:  term  l*j  <>r  P|C.  respectively .  in  e,|.».  (7)  -  (9)  is  .i 
non  factorable  function  of  six  iii<l>*|>cii<!nit  \itri.*I»!'*--.  that  tin*  optimization  problem 
is  a  composite  one.  For  numerical  i-iitliutiuns  it  is  recommended  In  hold  a  point 
P(x.y,z)  fixed,  perform  a  summation  with  re.*j»oct  1«»  fj  with  fixed  £.  tJ.  f  coordinates, 
integrate  over  the  £,  rj.  f  domain,  and  thereafter  integrate  with  respect  to  y.  •/..  A  nu¬ 
merical  analysis  and  a  subse«pn:nt  computer  program  does  not  present  formidable  diffi¬ 
culties.  The  integration  pertaining  to  z  ha*  to  start  at  zero  with  an  associated  cumula¬ 
tive  probability  of  the  distribution  5  (from  minus  infinity  to  zero). 

3.  Simplifications  am!  Ramifications  of  Six-Dimensional  Analytical  Approach. 
A  considerable  simplification  results  if  the  target  coordinate*  relate  to  a  center  of  grav¬ 
ity  easily  computed  through  the  use  of  standardized  target  areas  and  actual  target- 
density  descriptors  (indices).  Thereafter,  more  or  less  satisfactory  homogeneity  is  as¬ 
sumed  and,  in  general,  a  circular  target  area  which  may.  however,  vary  in  diameter. 

As  a  further  simplification,  the  terrain  is  assumed  to  be  negligible,  i.e.,  expressed 
only  by  an  average  height  d.  An  increased  ?“t>  or  S“ ~  would  satisfactorily  compen¬ 
sate  hereforc.  ’  *’* 


If  Sz  p  or  Sz*j'  is  relatively  small,  optimization  is  practically  two-dimensional. 
As  shown  in  fl70-2,,the  respective  patterns  are  invarient  under  a  rotation. 

In  Jhe  two-dimensional  case,  we  would  thus  have  optimal  coordinates  aj,  bj,  d: 
a2,  bo.  d;  ay.  by,  d.  etc.,  where  aj.  hj  may  also  he  C.  0.  Allhough  the  existence  of 
two  distributions  nvolving  the  random  v;.  sables  z  and  £  is  disregarded,  as  far  as  „ptnni- 
zatiou  is  concerned,  integration  with  respect  to  six  variables  is  still  nccessarv  in  order 
to  obtain  correct  parameters  for  the  carnally  probability  di.-tributions,  which  depend 
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on  Ilf  number  of  volievs  k  (lor  om-tant  rairje  ami  .t  given  set  of  target  descriptors 
such  as  size). 


The  three-dimensional  optimization  i*  required  milt  in  ease  of  a  greater  vertical 
mean  target  heigh!  uncertainty,  which  is  mostly  associated  with  longer  ranges,  includ¬ 
ing  missile  applications. 


Optimal  configurations  characterized  by  coordinate  triples  involving  different  d's 
might  then  appear  to  he  competitive,  such  as  0.  0.  tip  0.  0.  do  or  0.  0.  dj:  0.  0.  do; 
0,  0,  do:  ami,  in  exceptional  circumstances,  triples  reterring  to  the  first  optimization 

ring,  such  as  a K  hj^\  d b  a->^,  b-/^.  d . and  •. d./^: 

ao^®\  l»o^\  d./^: . Because  of  tlur  >v inmetries  involved,  the  optimization  struc¬ 

ture  is  well  discernible  and  the  ruuncricat-coinpiilationjl  effort  reduced.  It  should  he 
kept  in  mind  that  optimizations  and  evaluations  involving  the  vertical  geometrical  di¬ 
mensions  depends  on  fragment  damage  tables  for  various  heights  of  lmr»t. 


in  conclusion,  it  should  he  emphasized  that  a  considerable  reduction  of  computa¬ 
tional  work  is  expected  after  a  thorough  numerical  study  has  been  conducted  for  vari¬ 
ous  ranges,  larget  sizes.  number  of  volievs,  etc.,  for  :r  gii-  n  weapon  -y-tem.  The  p-Milts 
of  sueh  a  study  would  vich!  much  more  in-ight  into  weapon-  effects  and  their  po-.-ihle 
optimization  than  hitherto  avaihdde. 


Reproduced  from 
best  available  copy. 


281 

8est  Available  Copy 


A  MATHEMATICAL  MODEL  FOR  ADJUSTMENT  PHASE  PROCEDURES 
AND  MONTE  CARLO  INVESTIGATION  OF  THE  EFFECT  OF  THESE  PROCEDURES 
ON  FIRE  FOR  EFFECT  ROUNDS 


Sidney  Gerard 

Reliability  and  Maintainability  Division 
U.S.  Army  Materiel  Systems  Analysis  Agency 
Aberdeen  Research  4  Development  Center 
Aberdeen  Proving  Ground,  Maryland 

ABSTRACT 

The  development  of  n  mathematical  nodel  for  the  adjustment 
phase  procedures  utilized  by  artillery  and  morter  groups  and  a  Monte 
Carlo  simulation  of  these  adjustment  phase  procedures  is  presented 
herein  Investigations  on  the  number  of  rounds  required  to  accomplish 
the  adjustment  phase  of  the  mission  and  the  errors  of  the  centers  of 
impact  of  rounds  fired  for  effect  upon  accomplishment  of  the  adjust¬ 
ment  phase  are  also  presented  In  this  paper.  The  above  investiga¬ 
tions  are  dependent  on  many  factors  the  most  Important  being  the 
following: 

1.  First  round  aiming  errors. 

2.  Observer  to  target  distance. 

3.  The  angle  between  the  gun  to  target  (GT)  line  and 
observer  to  target  (OT)  line. 

4.  Target  size  (registration  area). 

5.  Round-to-round  dispersions. 

« .  Ooservcr  s  error  In  locating  impact  poluvs  v, 
adjustment  phase  rounds. 

Finally  a  sensitivity  analysis  has  been  conducted  on  these 
factors  and  results  of  the  significant  findings  presented  here. 


Most  of  this  paper  was  reproduced  photographically  from  the  manuscript 
supplied  by  the  author. 
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INTRODUCTION 


During  c  military  engagement  (battle)  where  Artillery,  mortar 
or  nuclear  weapon  systems  ire  being  employed  there  are  usually  four 
basic  methods  of  accuracy  fire  employed.  These  are  (1)  predicted, 

(2)  meteorological  plus  velocity  errors,  (3)  adjusted  and  (4)  K 
transfer  (includes  adjusted  lire)  fire.  However,  prior  to  use  of 
these  weapon  systems,  a  target  must  be  located  and  a  suitable  eleva¬ 
tion  and  azimuth  determined  which  will  enable  rounds  to  be  fired 
effect ively  on  the  target.  For  approximately  75X  of  the  accuracy 
fire  missions  the  determination  of  this  elevation  and  azimuth  Is 
accomnl isuod  by  adjustment  procedures.  Adjusted  Tire  is  the  method 
of  fire  in  which  n  number  of  rounds  arc  expended  and  through  successive 
corrections  the  suitflhle  elevation  and  arlmuth  are  obtained.  These 
corrections  are  directed  by  an  observer  usually  forward  of  the:  gun 
and  set  off  at  an  angle  less  than  +  5')0  mils  from  the  gun  to  target 
(GT)  line  of  fire. 

This  paper  presents  the  development  of  a  mathemati cal  model 
and  a  Monte  Carlo  simulation  of  those  adjustment  phase  procedures. 
Errors  in  the  center  oT  impact  of  rounds  fi-od  lor  effect  end  the 
number  of  rounds  expended  in  the  adjustment  phase  of  the  mission  are 
obtained  from  the  results  of  the  simulation. 
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ACCOMPLISHMENT  OF  THE  ADJUSTMENT  PHASE 


The  basic  objective  of  all  weapon  systems  la  to  fire  rounds  effec¬ 
tively  on  a  designated  target.  The  adjustment  phase  Is  the  part  of  an 
adjusted  fire  mission  in  which  a  number  of  rounds  arc  expended  and  through 
successive  corrections  a  suitable  elevation  and  atlmuth  are  obtained  to 
achieve  this  objective.  Questions  posed  concerning  the  use  of  this 
technique  of  fire  are:  (1)  what  is  the  expected  number  of  rounds  expended 
during  the  adjustment  phase  end  (2)  what  determines  the  final  suitable 
elevation  and  atlmuth.  The  adjustment  phase  Is  complete  and  the  above 
questions  answered  upon  fulfillment  of  any  of  the  three  conditions  below. 

1.  The  first  or  any  subsequent  round  during  the  adjustment  phase 

is  observed  to  impact  on  the  target  registration  a  ea  which  1b 
defined  by  a  predetermined  radius,  R  ,  from  the  enter  of  the 
target.  1 

2.  Ay  (usually  50,  100,  or  200)  meter  bracket  has  ^een  spilt  with 
reference  to  the  observer  to  target  (OT)  line.  Splitting  a  y 
meter  bracket  means  that  two  consecutive  rounds  Impact  within 

y  meters  of  each  other  and  bracket  (that  Is  one  round  impacting 
short  and  other  beyond  the  target  center)  the  center  of  the 
target.  However,  one-half  the  Indicated  correction  Is  applied 
before  firing  rounds  for  effect  upon  the  target. 

3.  The  Indicated  OT  range  correction  Is  less  than  or  equal  to 
and  the  OT  deflection  correction  Is  less  than  or  equal  to 

50  meters.  However,  the  deflection  correction  must  be  applied 
prior  to  firing  rounds  for  effect, 

ADJUSTMENT  PHASE  RANGE  AND  DEFLECTION  CORRECTIONS 

Corrections  made  b/  an  observer  during  the  adjustment  phase  are 
based  on  his  ability  to  estimate  the  Impact  points  of  the  rounds  fired. 
Bracketing  the  target  (that  Is  one  round  Impacting  short  and  another 
beyond  the  target)  on  the  first  two  rounds  of  the  mission  Is  a  critical 
factor  In  determining  the  number  of  rounds  required  to  accomplish  the 
mission.  During  an  observer's  training  he  Is  severely  penalised  if  he 
fall*  to  do  so  and  for  that  reason,  he  over  adjusts  prior  to  firing  the 
second  round.  Table  I  below  shows  minimum  observer  range  corrections 
depending  on  the  observer's  estimated  distance  to  the  target. 

The  observer  senses  the  round  impact  points  and  makes  his  corrections 
on  the  OT  line.  These  corrections  are  transformed  to  the  GT  line  and 
thus  enable  weapon  elevation  and  atlmuth  corrections  to  be  applied.  Atlmuth 
corrections  are  made  to  change  the  deflection  right  or  left  to  the  nearest 
ten  meters.  Since  range  is  much  more  difficult,  corrections  are  made  so 
that  the  elevation  Is  changed  to  increase  or  decrease  range  according  to 
Table  II. 
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Table  I 


Observer's  Estimated 
Distance  to  the 
Target  Center  (Meters) 

0  -  999 
1000  -  1999 
2000  -  over 

Table 

Observed  Impact  Point 
Deviation  (GT  line), 
from  Target  Center  (Meters) 


Minimum  Observer  Range 
Correction  Prior  to  Firing 
Second  Round  ^Meters) 

100 

200 

400 

II 

Amount  of 

>Sr 

Correction  (r  ) 
(Meters) 


Rt 

< 

A 

< 

100 

100 

100 

* 

A 

< 

200 

200 

200 

< 

A 

< 

300 

400 

300 

< 

A 

< 

500 

600 

500 

< 

A 

< 

700 

• 

600 

700 

< 

A 

< 

900 

1000 

900 

< 

A 

< 

m 

1200 

It  should  be  noted  that  if  A  is  positive  then  r  is  negative  and 
vice  versa. 


DEVELOPMENT  OF  A  MATHEMATICAL  MODEL  TO  REPRESENT 
THE  ADJUSTMENT  PHASE  PROCEDURES 


An  Adjusted  fire  mission  can  be  separated  into  two  basic 
phases  which  are  (l),thc  adjustment  and  (2)  fire  for  effect  phases. 
This  paper  is  conceA«il-vv|’*ii \\he  adjustment  phase  effects  on  the 

v.  • 

fire  for  efface. ->haae 

\ 

7  / 

•  x 

a  .  v 

The  first  r.tcp  in  any  «\c  justment  phase  is  to  aim  the  weapon 

'  >  > 

at  the  desired  target  'i  «.  ft.  an  elevation  and  azimuth  to  fire 
the  first  round  of  the  riscion).  Since;  J. i #t  .'early  impossible  to 


ala  any  weapon  system  on  the  center  iht-  .tJ'  get,  aiming  errors 

are  inherent  in  all  firing  missions.  The  aiming  errors  involve  (1) 

meteorological  (2)  target  location  and  (3)  other  related  predicted 

fire  errors  (such  as  estimated  velocity,  elevation,  and  azimuth 

measurements,  etc.).  However,  this  paper  is  concerned  with  adjusted 

fire  missions  only  and  thus  aiming  errors  only  affects  die  first 

round  of  every  Individ  lal  adjustment  phase.  Since  Che  range  aim 

point  (y)  has  true  range  (y_)  and  stendard  deviation,  o_,  and  the 

x  y 

deflection  aim  point  (x)  has  expected  deflection  (Uj,)  and  standard 
deviation,  o_,  some  method  of  characterizing  the  inaccuracies  of 
estimating  the  above  expectations  must  be  devised.  It  is  assumed 
that  the  expected  range  and  deflection  follow  some  arbitrary  dis¬ 
tribution  and  therefore  the  density  of  impacts  at  ranges  y  and 
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deflection  x^  can  be  expressed  by  the  functions,  p(y:py,  Oy)  and 
p(x:ur,  oc),  respectively. 

Upon  having  aimed  che  weapon  system  the  first  round  is  fired 
end  If  the  system  had  perfect  accuracy  the  round  impact  point  co¬ 
ordinates  would  be  (y,x).  Since  no  weapor  <*/8ioma  have  been  devised 
with  perfect  accuracy,  some  means  of  characterising  the  inaccuracies 
must  be  developed.  It  is  assumed  that  the  range  delivery  errors 
follow  some  arbitrary  distribution,  p(y:y,  oy)  where  y  is  the  ex¬ 
pected  range  and  o  is  the  round-to-round  range  standard  deviation. 

/ 

Similarly,  the  delivery  errors  existing  in  deflection  can  be  expressed 
by  the  distribution  p(x:x,  o^) . 

The  coordinate  of  the  actual  impact  point  for  the  a dj us tr.se at 
phase  round  is  represented  by  (yj,  Xj)  with  respect  to  the  CT  line  or 

»  i 

(y^,  *1^  w-th  respect  to  the  OT  line.  Since  the  observer  utilises 
his  estimate  of  (y^,  x^)  to  make  OT  line  deviations  (range)  and 
(deflection)  an  error  in  the  adjustment  phase  is  attributable  to  his 
alslocstlon  of  the  Impact  point.  A  method  must  be  devised  to 
characterise  these  errors.  It  is  assumed  that  his  range  errors,  ye, 
can  be  expressed  by  aome  arbitrary  distribution,  p(ye:Uye,  ay6>  and 
the  observer's  deflection  errors,  xe,  are  expressed  by  p(xc:pxe,  o^) 
and  are  similarly  characterised  in  each  subsequent  adjustment  phase 
round. 


If  none  of  the  previously  defined  conditions  necessary  for 
accomplishment  of  the  adjustment  phase  have  been  satisfied  the 
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corrections  r^  and  for  range  and  deflection  respectively  are 
applied  and  the  second  round  is  fired.  Hie  distribution  of  the 
delivery  errors  associated  with  an  individual  weapon  system  does  not 
vary,  however,  due  to  the  application  of  the  above  corrections  the 
expected  range  and  deflection  for  the  second  round  are  y+r^  and 
x+d^  respectively.  The  distributions  of  range  and  deflection  for 
the  second  round  are  expressed  by  the  functions,  p(y:y+r^,  o^)  and 
p(x:x+d^,  °x) ,  respectively.  In  a  similar  manner  the  expected 
range  and  deflection  impact  points  changes  for  each  subsequent  ad¬ 
justment  round  depends  on  the  corrections  applied.  For  the  i^ 
round  of  any  adjustment  mission  the  expected  range  and  deflection 
can  be  expressed  by 

Rnn  p.e ; 


7 

For  i  -  1 

(1) 

i-l 

y  +  £  rk 
k-1 

For  1  >  1 

(2) 

Deflection: 

»  X  For  i  -  1  (3) 

i-1 

x  -  x  ♦  £  \  For  i  >  1  (4) 

1  k-1  K 

where  y  ans  x  are  the  expected  range  and  deflection  for  the  first 
round  of  an  adjustment  mission  anj  the  second  terms  of  each 
expression  are  total  range  and  deflection  corrections  applied 
before  firing  the  i1*1  round. 
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Since  all  of  these  previously  described  errors  affect  any 
adjustment  mission  the  question  posed  upon  completion  of  the  adjust¬ 
ment  phase  is  that  of  the  center  of  impact  of  rounds  fired  for  effect. 
The  ultimate  goal  of  the  adjustment  phase  or  any  method  fire  is  to 
obtain  an  elevation  and  azimuth  which  will  have  an  aim  point  with  the 
coordinates  (p^,  Px)  where p  is  che  true  range  from  weapon  system  to 
target  center  and  there  is  no  deflection  error.  However,  this  is  a 

practical  Impossibility  and  upon  completion  of  the  adjustment  phase 

*  * 

the  rounds  fired  for  effect  will  have  aim  point  coordinates  (y  ,  x  ) 

A  A 

where  y  and  x  are  the  range  and  deflection  centers  of  impact  of 
these  rounds, 

Because  it  was  assumed  that  range  and  deflection  errors  ap 

A  A 

independently  distributed  it  is  possible  to  work  with  y  and  \ 
individually.  The  adjustment  phase  is  terminated  if  condition  (1) 
any  round  is  observed  to  impact  on  the  target  registration  area  or 
(2)  a  y  meter  bracket  has  been  split  or  (3)  the  OT  line  range 
correction  is  _<  Rt  and  the  OT  line  deflection  correction  is  50 
meters  is  met.  Assuming  the  adjustment  phase  is  terminated  on  the 
n*"*1  round  then  using  equations  (1),  (2),  (3),  and  (4)  riie  expected 
range  and  deflection  for  this  round  are 
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Range : 


9 

For  n  =  1 

(5) 

n-l 

y  +  zl  r 

k=l  k 

For  n  >  1 

(6) 

Deflection : 


X 

For  n  *  1 

(7) 

n-l 

X 

+  £  dk 

For  n  >  1 

(8) 

k»l  k 

If  termination  of  the  adjustment  phase  was  due  to  fulfillment 
of  conditions  (1)  or  (3)  the  deflection  corrections  are  applied  and 
the  fire  for  effect  phase  is  begun  thus  the  center  of  impact  o!  these 
rounds  will  be  the  following 


Rar.y.e : 

*  _ 

y  ”  y  «=  v  For  n  ®  1 

^  n  - 


it  _  ‘t-t 

y  «  y  -  y  +  2^  rK  For  n  >  1 
k*l  K 


Deflection : 


* 

x  'X 

n 


(9) 

(10) 


<U) 


However,  if  condition  (2)  is  met  the  deflection  center  of  impact  remains 
as  shown  by  equation  (11)  but  since  half  the  indicated  ->nge  correction 
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is  applied  the  range  cencer  of  impact  of  the  fire  for  effect  phase 
rounds  is  now  defined  by 

y  ■  y  +  1/2  for  n  ■  1  (12) 

*  n"1 

y  -  9  +  T,  ri  ♦  1/2  r  For  n  >  1  (13) 

iTTl  n 

Thus  equations  (9),  (10),  (12),  or  (13)  i>  the  mathematical 
representation  of  the  range  center  of  impact  of  the  fire  for  effect 
phase  rounds  and  equation  (11)  is  the  deflection  representation  of 
these  rounds. 

*  * 

It  is  apparent  that  estimates  of  the  parameters  y  ,  x  ,  n, 

o  *,  o  *  ,  and  a  are  a  necessity  in  determining  the  effects  the 
y  x  n 

adjustment  phase  have  on  the  fire  for  effect  phase. 
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ESTIMATES  OF  THE  ADJUSTMENT  PHASE  PARAMETERS  RASED  ON 
THE  NORMAL  DISTRIBUTION 

A  method  had  to  be  devised  to  obtain  estimates  of  these 
parameters  due  to  the  fact  that  there  was  a  lack  of  data  and  extreme 
difficulty  in  conducting  experiments  to  obtain  this  data.  It  was 
decided  that  Monte  Carlo  techniques  would  be  used  to  obtain  estimates 
of  these  parameters. 

Prior  to  using  these  Monte  Carlo  techniques  it  was  necessary 
to  assume  that  the  (1)  aiming,  (2)  delivery  and  (3)  observer  errors 
previously  discussed  follow  some  particular  distribution  or  distribu¬ 
tions.  For  the  purpose  of  this  paper  these  errors  were  assumed  to  be 
normally  distributed. 

Since  the  aiming  errors  were  assumed  to  be  normally  distribu¬ 
ted,  the  density  of  impacts  at  ranges  y  j  and  deflection  can  be 
expressed  by  the  functions 
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where  p-  ami  u«-  are  expected  range  and  deflection,  and  o-  and  o~  are 
y  x  y  x 

the  standard  deviations  of  the  aiming  errors  associated  with  range  and 
deflection,  respectively. 

The  second  type  of  error  associated  with  the  adjustment  phase 
rounds  is  the  delivery  error.  The  distributions  of  ra..ge  and 
deflection  can  be  expressed  by  the  functions 


The  third  error  associated  with  the  adjustment  phase  rounds 
i9  the  observer  error  in  locating  impact  points.  Since  it  was  assumed 
his  errors  arc  normally  distributed,  the  density  of  the  observer 
errors  in  range,  ye,  and.  deflection ,  xe ,  can  be  expressed  by  the 
functions. 


29  U 


p(ye) 


1 


where  w  and  u  are  the  expected  observer  errors  and  o  and  o  ore 
ye  xe  r  ye  xe 

the  round-to-round  standard  deviations  in  observer  errors  rnr>f,c 
and  deflection,  respectively. 
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MONTE  CARLO  TECHNIQUES 


In  order  to  use  Monte  Carlo  techniques,  it  was  necessary  to 
draw  random  samples  from  normal  distributions.  The  Ballistic  Research 
Laboratories  Electronic  Scientific  Computers  (BP* ESC  I  and  II)  at 
Aberdeen  Proving  Ground,  Maryland,  were  used  to  generate  those 
random  numbers. 


A  random  number,  Z,  was  generated  from  a  normal  distribution 
with  a  mean,  0,  and  standard  deviation,  1,  i.e.,  N(0,1).  This 

normal  random,  number,  Z,  was  then  converted  to  a  normal  random 

* 

number,  Z  ,  from  N  o^*).  As  30  example  let  us  consider  the 

sampling  o.  the  range  aim  point  for  the  first  adjustment  round, 


which  is  known  to  have  mean,  Uy,  (assumed  to  be  the  actual  target 
center)  and  standard  deviation,  o^..  A  normal  random  number, 

Z,  from  a  N(0,1)  is  generated  and  converted  to  a  normal  random 


number,  Z  ,  from  a  N  (y_,  a_)  .  This  procedure  for  generating  aim 
points,  impact  points  and  observer  errors  was  used  throughout  these 


adjustment  procedures. 


Estimates  of  the  means  and  standard  deviations  of  y  and  x 
for  samples  of  size  5000  were  generated  using  the  above  procedures. 
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A  PRACTICAL  APPLICATION  FOR  THE  RESULTS  GENERATED 


The  broad  problem  of  gun  tube  life  termination  involves  the 
study  of  several  criteria  to  establiah  the  controlling  factors  which 
correspond  to  the  lowest  limit  in  terms  of  number  of  rounds  corre¬ 
sponding  to  acceptable  performance  of  a  gun  system.  A  major  factor 
in  determining  gun  tube  life  termination  is  accuracy-cost.  Accuracy- 
cost  can  be  best  described  as  such,  the  accuracy  of  a  gun  system  Is 
degraded  as  the  number  of  rounds  fired  from  a  tube  increases.  The 
wear  characteristics  of  the  tube  cause  greater  round-to-round  dis¬ 
persion.  As  the  number  of  rounds  fired  from  a  tube  Increases  there 
Is  a  balance  between  the  tube  cost  per  round  and  ammunition  costs  as 
the  number  of  rounds  required  to  hit  a  target  increases.  A  point 
is  reached  where  the  increased  anmunition  costs  more  than  compensate 
for  the  tube  cost  savings  effected  by  extending  the  tube  life. 

A  method  for  determining  gun  tube  life  termination  based  on 
accuracy-cost  was  developed  by  Bruno,  Kniss  and  Gerard  (1).  This 
method  involves  the  concept  of  an  effective  round  which  is  defined 
as  a  round  which  hits  a  target  during  either  the  adjustment  or  fire 
for  effect  phases. 
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The  probability  of  an  effective  round  was  defined  13 


P(effective  round)  -  C(l-A)  +  A(B)  (20) 

where  A  is  the  ratio  of  che  rounds  used  in  an  adjustment  phase  to  the 

total  number  of  rounds  fired  In  a  mission  (except  for  emergency 

missions  the  number  of  rounds  fired  are  generally  predetermined),  B 

is  the  probability  of  a  round  hitting  the  target  during  the  adjustment 

phase  and  C  is  the  probability  of  a  round  hitting  the  target  during 

the  fire  for  effect  phase.  Estimates  of  the  mean  number  of  rounds 

expended  during  the  adjustment  phase  were  used  to  obtain  estimates  of 

A  and  o  *  and  a  *  were  used  to  determine  the  overall  dispersion  of 
y  x 

the  given  weapon  system  in  obtaining  estimates  of  C. 

The  ammunition  cost  per  effective  round  was  then  defined  as 

Y  "  C(l-A)+A(b)  (21) 

where  S  is  the  ammunition  cost  per  round  ana  the  denominator  of 
equation  (21)  is  the  probability  of  an  effective  round  as  previously 
defined  by  equation  (20). 

By  utilizing  equations  (20  and  (21)  and  results  of  these  Monte 
Carlo  simulations  optimum  gun  tube  life  termination  round  numbers 
based  on  accuracy-cost  could  he  determined  for  several  gun  systems. 
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SENSITIVITY  ANALYSIS 


Upon  having  developed  a  mathematical  model  to  represent 
these  adjustment  phase  procedures,  Monte  Carlo  techniques  were  used 
to  generate  data  to  obtain  estimates  of  the  mean  and  dispersions  for 
three  basic  parameters  associated  with  these  procedures.  These 
parameters  were  (1)  the  number  of  rounds  required  to  accomplish  an 
adjustment  mission,  (2)  the  range  center  of  impact  and  (3)  the 
deflection  center  of  impact. 

Prior  to  conducting  a  sensitivity  analysis,  investigations 
to  determine  which  factors  would  possibly  effect  these  adjustment 
phase  procedures  were  conducted.  Since  these  investigations 
revealed  there  were  numerous  factors  their  minimum  and  maximum 
estimates  were  used  to  investigate  their  effect  on  the  adjustment 
phuse  procedures.  For  example  data  was  generated  for  deflection 
round-to-round  standard  deviations  of  1  and  20. 

The  sensitivity  analysis  was  conducted  to  determine  the 
effects  of  seven  factors  on  the  desired  parameter  estimates.  The 
first  and  second  factors,  target  sire  (registration  area)  and  range 
round-to-round  dispersion,  significantly  effected  the  results  ob¬ 
tained  for  all  the  above  parameters.  The  third  factor  investigated 
was  the  deflection  round-to-round  dispersion.  Results  indicated 


?99 


that  Increasing  the  deflection  round-to-round  standard  deviation 
resulted  in  a  significant  increase  of  the  deflection  center  of  impact 
dispersions,  but  had  no  effect  on  the  estimates  of  the  remaining 
parameters. 

Factor  four,  the  angle  between  tl  r  lire  and  OT  line, 
significantly  effected  the  dispersion  estimates  of  the  renge  and 
deflection  center  of  impacts.  Tire  range  estimates  were  significantly 
decreased  while  the  deflection  estimates  were  significantly  increased 
as  the  angle  was  Increased.  The  fifth  factor,  OT  distance  did  not 
significantly  effect  the  parameter  estimates,  but  significantly 
effects  the  observer's  errors  in  locating  impact  points  of  adjust¬ 
ment  phase  rounds,  the  sixth  factor.  The  estimates  of  observer 
errors  used  <n  this  paper  were  based  on  the  miss  distance  of  the 
impact  point  from  the  target  center  which  was  weighted  according  to 
the  OT  distance.  These  estimates  were  based  on  judgment  only  due 
to  the  lack  of  available  data.  However,  efforts  are  currently  being 
made  to  conduct  experiments  to  generate  data  to  estimate  the 
observer's  errors.  It  should  be  noted  that  these  adjustment  phase 
procedures  arc  sensitive,  depending  on  the  magnitude,  to  these 
observer  errors.  However,  it  is  felt  that  data  from  the  alove 
experiments  will  probably  not  yield  observer  errors  of  sufficient 
magnitude  to  effect  the  results.  Ihc  final  factor,  the  first  round 
aiming  errors,  had  no  practical  significance  on  the  results. 


300 


Although  the  sensitivity  snslysis  dsts  could  bt  utilised  to  obtain 
some  optimum  policy  for  those  adjustment  phase  procedures,  this  optimum 
policy  could  not  be  schleved  since  moat  of  the  factors  are  uncontrollable, 

l.a.,  such  as  the  observer's  ability  to  control  his  angle  or  distance  to 
the  target  because  of  environmental  factors. 

Results  of  the  data  generated  by  the  Monte  Carlo  techniques  are 
summarised  In  Tables  1  through  VI 

CONCLUSIONS 

It  has  bean  shown,  ty  f'ertorelng  a  sensitivity  analysis  on  data 
obtained,  that  six  of  the  seven  factors  Investigated  effected  the  estimates 
of  the  parameters  of  these  adjustment  phase  procedures.  However,  these 
parameter  estimates  have  been  shown  to  be  more  sensitive  to  the  target 
slse  (registration  area)  and  the  rouno-to-round  range  dispersion. 

These  results  Indicate  that  employing  adjusted  fire  techniques  the 
cMsperslon  of  che  fire  for  effect  rounds  is  smaller  than  for  any  other 
method  of  fir*.  Using  these  results  It  has  been  shown  the  probabilities 
of  hitting  a  gvvvn  target  Is  greater  for  adjusted  fire  (even  including 
\  'he  rounds  for  the  adjustment  phase)  than  for  any  other  method  of  fire. 

\  Therefore,  these  results  strengthens  the  artillerymen's  position  of 
>  e>  iloylng  adjusted  fire  whenever  possible. 

Furthermore,  estimates  of  these  parameters  are  being  used  to  Improve 
current  estimates  of  the  overall  accuracies  and  optimum  tube  life  of 
weapon  ayeceme. 
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APPENDIX  A 


TABLES  OF  KEANS  AND  DISPERSIONS  OF  THE  NUMBER 
OF  ROUNDS  EXPENDED  DURING  AN  ADJUSTMENT  MISSION 

TABLE  III 

Main  and  Diaparslona  of  Cha  Number  of  Rounds  Expended 
during  an  Adjustment  Phase  Mission  For  a  Target  of  Radius  1  Meter 

0^  ■  25  meters 


Deflection 

Obaorver* 

_ _ 

OT  Anel 

s  (Radians) 

0 

1  3  1 

Rd.-to-Rd. 

Standard 

,  Average 

Std.  Dev. 

Average 

Std.  Dev, 

Std.  Dev. 

Deviations 

No.  of  Rds. 

of  No.  of 

No.  of  Rds. 

of  No.  of 

(ft) 

Rds.  (c  ) 
n 

'.ft) 

Rds .  (o  ) 
n 

OK 

3.74 

1.44 

4.09 

1.67 

1 

“a 

°E3 

5.64 

4.10 

2.54 

1.75 

6.74 

4.75 

2.91 

2.15 

7.84 

3.65 

8.58 

3.65 

3.99 

1.57 

■SHI 

1.76 

5.86 

2.69 

■EH 

2.94 

20 

4.23 

1.92 

ms 

2.15 

7.99 

3.77 

a.  74 

3.68 

50  meters 


*  The  observer  Standard  Deviations  are  defined  In  footnotes  St  the 
end  of  this  article. 
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TABLE  lllfCont’d) 

Mean  and  Dispersions  of  the  Number  of  Rounds  Expended 
during  an  Adjustment  Phase  Mission  For  a  Target  of  Radius  1  Meter 


100  meters 


Deflection 
Rd.-to-Rd. 
Std.  Dev. 
(o  ) 


|  OT  AnRle  (Radians 

Observe  re  j  0  ' 

Standard  |  Average  jStd.  Dev.  Averag 


Deviations 

(o  So) 
ye  xe 


Rds .  (o  ) 
n 


Average  Std.  Dev.  Average  Std.  Dev. 
No.  of  Rds.  of  No.  of  No.  of  Rds.  of  No.  of 


Rds.  (cO 


6.<.r 

3.54 

7.36  j 

3.61 

6.73 

3.62 

8.06  ‘ 

3.91 

*  The  observer  Standard  Deviations  are  defined  in  footnotes  on  Page  326 
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TABLE  III (Cont’d) 

Mean  and  Dispersions  of  the  Number  of  Rounds  Expended 
during  an  Adjustment  Phase  Mission  For  a  Target  of  Radius  1  Motor 


o  ■  200  meters 

y 


Deflection 
Rd.-to-Rd. 
Std.  Dev. 

<v 

Observer* 

Standard 

Deviations 

(o  6  o  ' 

ve  xc 

0T  An„le 

(Radiam) 

.  .  J 

ihhhhkb 

Average 

No.  of  Rds. 

(n> 

Std.  Dev. 

of  No.  of 

Rds.  (o  ) 
n 

Average 

No.  of  Rds. 
(n) 

Std.  Dev. 

of  No.  of 

Rds .  (a  ) 
n 

0E1 

5.42 

mm 

7.30 

3.82 

oe2 

5.77 

Hfc: 

7.63 

3.79 

1 

oe3 

5.29 

3.07 

7.  36 

3.85 

oe4 

.7.34 

3.83 

7.89 

3.87 

II 

5.84 

3.  34 

7.27 

3.80 

sit.--/ 

6.32 

3.52 

7.91 

3.81 

20 

mm 

5.59 

3.24 

7.25 

3.79 

7.56 

3.92 

8.14 

3-91 

*  The  observer  Standard  Deviations  are  defined  in  footnotes  on  Page  326 
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TABLE  II 

Mean  and  Dispersions  of  the  Humber  of  Rounds  Expended 
during  an  Adjustment  Phase  Mission  For  a  Target  Radius  of  25  Meters 


o  •  25  meters 
y 


1 

Deflection 

Observer* 

OT  Ancle 

Radians) 

0 

Rd.-to-Rd. 
Stdj  Dev. 

<o  ) 

X 

Standard 

Deviations 

<\e  *  \e> 

Average 

No.  of  Kds. 
(n) 

Std.  Dev. 

of  No.. of 

Rds .  (a  ) 
n 

Average 

No.  of  Rds. 
(n> 

Std.  Dev. 

of  No. .of 

Rds.  (o _) 
n 

0E1 

3.32 

1.19 

3.37 

1.26 

«, 

4.55 

2.10 

5.03 

2.15 

1 

OE. 

3.50 

1.39 

3.76 

1.58 

6.81 

3.76 

7.27 

3.94 

B  ■ 

3.35 

1.17 

3.40 

1.30 

4.60 

2.10 

5.20 

2.40 

20 

B 

3.53 

1.43 

3.78 

1.67 

i  •  b 

6.95 

3.76 

7.45 

3.81 

*  The  observer  Standard  Deviations  are  defined  in  footnotes  on  Papa  326 
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TABLE  II  (Cont'd) 

Mean  and  Dispersions  of  the  Number  of  Rounds  Expended 
during  an  Adjustment  Phase  Mission  For  a  Target  Radius  of  25  Meters 


o  “  100  meter9 

y 


Deflection 
Rd.-to-Rd. 
Std.  Dev. 

<8x> 

Observer* 

Standard 

Deviations 

(3  io) 
ye  xe 

0T  Ancle 

(Radians) 

J  "  ‘  1  1 

o 

_  .4 

Average  i Std.  Dev. 

No.  of  Rds.|of  No.  of 
(n)  ]  Rds .  (o  ) 

_ .  n 

Average 

No.  of  Rds. 
(n) 

Std.  Dev. 

of  No., of 

Rds.  (a  ) 
n 

°E 

3.48 

1.57 

4.01 

2.12 

4.33 

2.23 

5.30 

2.82 

1 

0E3 

3.60 

1.75 

4.26 

2.24 

oe4 

6.45 

3.69 

7.24 

3.87 

B  B 

3.60 

1.68 

4.02 

2.12 

4.44 

2.28 

5.35 

2.82 

20 

fl  - 

3.75 

1.89 

4.29 

2.31 

6.60 

3.75 

7.22 

3.89 

OEx 

4.17 

2.27 

5.16 

3.04 

1 

OE, 

4.56 

2.50 

5.89 

3.24 

«, 

4.12 

2.32 

5.15 

3.02 

6.52 

3.78 

7.25 

3.89 

0E1 

4.39 

2.46 

5.19 

3.08 

0E2 

4.87 

2.  79 

5.86 

3.22 

20 

oe3 

4.28 

2.  39 

5.25 

3.05 

oe4 

6.58 

3.82 

7.16 

3.93 

*  The  observer  Standard  Deviations  are  defined  in  footnotes  on  Page  326 
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TABLE  II  (Cont'd) 

Meen  and  Dispersions  of  the  Number  of  Rounds  Expended 
during  an  Adjustment  Phase  Mission  For  a  Target  Radius  of  25  Meters 


o  ■  200  Meters 

y 


Deflection 
Rd.-to-Rd. 
Std,  Dev. 

(0x) 

Observer* 

Standard 

Deviations 

(%e  4  \e> 

0T  Angle 

(Radians  1 

— 

0 

« 

4 

Ave rage 

No.  of  Rds. 

(S) 

Std.  Dev. 

of  No. ^of 

Rds .  (c  ) 
n 

Average 

No.  of  Rds. 
(n) 

Std.  Dev. 

of  No.^of 

Rds.  (o  ) 
n 

OKi 

4.58 

2.89 

6.07 

3.52 

0*2 

4.90 

2.86 

6.25 

3.49 

1 

OE. 

4.57 

2.74 

5.87 

3.42 

0E< 

6.61 

3.82 

7.28 

4.00 

D 

5.10 

3.03 

6.04 

3.52 

5.20 

3.05 

6.25 

3.52 

20 

Kfl 

4.90 

2.91 

5.74 

3.38 

■9 

6.75 

— 

3.89 

7.22 

3.98 

*  The  observer  Standard  Deviations  are  defined  in  footnotes  on  Page  326 
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TABLE  III 

Mean  and  Dispersion  of  Che  Number  of  Rounds  Expended 
during  an  Adjustment  Phase  Mission  for  a  Target  of  Radius  100  Meters 


25  meters 


Deflection 

Rd.-to-Rd. 

Std.  Dev. 

(6  ) 

X 

^  OT  Angie  (Radians)  1 

Observer* 

Standard 

Deviations 

(o  So) 
ve  xe 

0 

.*  1 

Average 

No.  of  Rds. 
(H) 

Std.  Dev. 

of  No,  of 

Rds.  (o  ) 
n 

Average 

No.  of  Rds. 
(n) 

Std.  Dev. 

of  No. ^of 

Rds.  (o  ) 
n 

0E1 

2.74 

1.07 

2.55 

0.99 

3.24 

1.75 

3.36 

1.83 

1 

2.79 

1.26 

2.80 

1.29 

5.08 

3.58 

5.34 

3.61 

OE 

2.76 

1.06 

2.61 

1.01 

OE 

3.22 

1.72 

3.38 

1.84 

20 

i, 

OE. 

2.72 

1.21 

2.80 

1.30 

«. 

5.14 

3.56 

5.25 

3.59 

o  ■  50  meters 

y 


°E 

2.84 

1.21 

2.68 

3.24 

1.84 

3.48 

1 

°e3 

2.83 

1.36 

2.85 

1.40 

] 

0E. 

4 

5.13 

3.60 

5.39 

3.64 

j 

oo 

CM 

i  I 

1.19 

1.14 

i 

3.30 

1.85 

SSI'- 

20 

2.84 

1.37 

n 

5.10 

3.50 

WEM 

3.67 

*  The  observer  Standard  Deviations  are  defined  in  footnotes  on  Page  326 
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TABLE  III  (Cont'd) 

Mean  and  Dispersion  of  the  Number  of  Rounds  Expended 
during  f.n  Adjustment  Phase  Mission  for  a  Target  of  Radius  100  Maters 


o 

y 


100  meters 


OT  Ancle  ^Radians! 

_ 

Deflection 

Observer* 

>  _ 

5  1 

Rd  •  *  • 

Standard 

Average 

Std.  Dev. 

Average 

Std.  Dev. 

Std.  Dev. 

Deviations 

No.  of  Rds . 

of  No.  of 

No.  of  Rda. 

of  No.  of 

<°x> 

(o  &  a  ) 

ve  xe 

(R) 

Rna.  (o  ) 
n 

(n) 

Rds.  (6  ) 
n 

*1 

2.81 

1.31 

2.76 

1.33 

OE. 

3.19 

1.83 

3.41 

1.99 

1 

Hp 

2.73 

1.39 

2.88 

1.47 

5.07 

3.55 

5.25 

3.63 

0E1 

2.80 

1.29 

2.78 

1.34 

0B2 

3.13 

1.76 

3.40 

1.94 

20 

« 

2.78 

1.  AO 

2.90 

1.51 

«* 

A. 93 

3.48 

5.24 

3.66 

*  The  observer  Standard  Deviations  are  defined  In  footnotes  on  Page  326 
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TABLE  III  (Cont'd) 

Mean  and  Dispersion  of  the  Number  of  Rounds  Expended 
during  an  Adjustment  Phaae  Mission  for  a  Target  of  Radius  100  Meters 


o  ■  200  meters 

y 


Deflection 
Rd.-to-Rd. 
Std.  Dev. 

<°x> 

Observer* 

Standard 

Deviations 

(a  i.  o  ) 
'  ye  xe 

r* - 

0T  Ancle 

(Radians) 

1 _ Q _ 

4 

Average 

No.  of  Rds. 

(«> 

Std.  Dev. 

of  No.  of 

Rds .  (o  ) 
n 

Average 

No.  of  Rds. 
(«) 

Std.  Dev. 

of  No.  of 

Rds.  (o  ) 
n 

081 

3.67 

2.31 

3.71 

2.37 

3.45 

2.26 

3.83 

2.52 

1 

3.30 

2.05 

3.48 

2.15 

5.09 

3.60 

5.39 

3.77 

°E1 

3.75 

2.36 

3.78 

2.44 

0E, 

3.58 

2.33 

3.76 

2.44 

20 

«, 

3.35 

2.09 

3.57 

2.28 

5.14 

3.70 

5.42 

3.85 

*  The  observer  Standard  Deviations  are  defined  in  footnotes  on  Page  326 
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TABLE  IV  (Cont'd) 

Mean*  and  Dlapcrslons  of  Range  and  Deflection 
Canter  of  Impacts  For  a  Target  of  Radius  1  Meter 

Range 

0^  ■  SO  meters 


Deflection 
Rd.-to-Rd. 
Stdt  Dev. 

<9x> 

Observer* 

Standard 

Deviation 

<°ve  *  ®xe> 

0T  Angle 

(Radians) 

— 

0 

• 

Average 
Center 
of  Impact 

Standard 
Deviation 
of  Cl’a 

Average 
Center 
of  Impact 

Standard  [ 
Deviation 
of  Cl's 

0E1 

-4.68 

48.19 

0.69 

43.96 

or2 

-3.48 

138.96 

-0.18 

69.65 

1 

«, 

-11.25 

67.12 

-1.46 

50.04 

0E4 

-8.29 

3b9 .06 

1.47 

131.12 

!  OBj 

-2.13 

46.85 

1.69 

46.46 

OE 

-4.19 

121.85 

-0.62 

60.92 

20 

«, 

-7.26 

61.91 

-1.09 

49.47 

oe4 

-15.57 

306.11 

6.87 

102.23 

Deflection 


*  TTie  observer  Standard  Deviations  are  defined  In  footnotes  on  Page  326 


\ 


i 

J 

i 

1 

3 


312 

j 

I 

t 


a 


i 


liirl 


TABLE  IV  (Cont'd) 

Means  and  Dispersions  of  Range  and  Deflection 
Center  of  Impacts  For  a  Target  of  Radius  1  Meter 

Range 

0^  •  100 meters 


Deflection 
Rd.-to-Rd. 
Std.  Dev. 

<‘x> 

Observer* 

Standard 

Deviation 

(o  bo) 
ve  xe 

_ 0T  Angle 

wumsmi 

_ Q _ 

^  i 

Average 
Center 
of  Impact 

Standard 
Deviation 
of  Cl’s 

Ave  rage 
Center 
of  Impact 

Standard 
Deviation 
of  Cl’s 

OR 

-17.01 

112.71 

-3.25 

99.38 

0S2 

-16.46 

207.18 

-0.19 

131.95 

1 

<*, 

-28.24 

139.57 

-9.05 

104.40 

0E4 

-0.99 

415.32 

13.01 

185.87 

0E1 

-16.61 

110.16 

-2.94 

99.65 

°E 

-19.68 

210.51 

“10.85 

129.69 

20 

<* 

-28.26 

136.29 

-6.99 

108.57 

2a 

-27.08 

U— — _ 

453.10 

2.22 

186.91 

Deflection 
a  *100  meters 

y 


1 

°Ei 

OE2 

°E3 

0E4 

■ 

2.97 

5.05 

3.22 

12.76 

6.56 

11.51 

10.12 

5.79 

30.35 

53.64 

38.88 

85.32 

0E1 

MEM 

BB 

5.95 

34.21 

°e2 

H99 

■99 

13.22 

58.10 

20 

oe. 

0.29 

17.98 

8.49 

43.35 

«. 

0.53 

26.30 

9.42 

84.65 

*  The  observer  Standard  Deviations  are  defined  in  footnotes  on  Page  326 
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TABLE  IV  (Coin'd) 

Means  and  Dispersions  of  Range  ana  Deflection 
Center  of  Inpacca  For  a  Target  of  Radius  1  Meter 


Range 

0^  ■  ISO  meters 


Deflection 
Rd.-to-Rd. 
Std.  Dev. 

<°x> 

Observer* 

Standard 

D§viati$n 

<Cve  6  V* 

OT  Arele 

"~7K «<7T  ns3 

0 

rt  .rY 

Average 
Center 
of  Impact 

Standard 
Deviation 
of  Cl's 

Averu  ;e'  biin  >ird 

Cento  I'^via'.iyn 

of  Impact  of  ci'a 

0E1 

-13.90 

79.64 

0.54 

70.57 

oe2 

-13.11 

176.90 

-1.28 

100.50 

1 

0E3 

-22.51 

108.40 

-3.63 

79.55 

0E. 

4 

-15.35 

379.91 

7.93 

155.59 

0E! 

-11. 36 

80.01 

-2.04 

72.00 

OF., 

-11.57 

167.31 

106.66 

20 

0E3 

-18.00 

104.82 

81.02 

oe4 

-4.29 

360.70 

10.78 

158.12 

Deflection 

6  *  150  meters 

y 


1 

0E1 

oe2 

0E3 

oe4 

n 

2.97 

4.97 

3.56 

12.38 

5.99 

8.58 

7.76 

6.61 

28.60 

47.43 

35.10 

80.76 

0E1 

0.56 

17.81 

5.11 

32.90 

OE, 

0.49 

18.76 

9.36 

51.54 

20 

- 

-0.31 

18.01 

7.32 

39.38 

-0.25 

26. 38 

7.86 

81.48 

*  The  observer  Standard  Deviations  are  defined  in  footnotes  on  Page  326 
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TABLE  IV  (Cont'd) 

Means  and  Dispersions  of  Range  and  Deflection 
Center  of  Impacts  For  a  Target  of  Radios  1  Meter 


Range 

* 

0^  ■  200  meters 


Deflection 
Rd.-to-Rd. 
Std.  Dev. 

{ox> 

n 

Observer* 

Standard 

Deviation 

(°ve  4  °xe> 

0T  Annie 

(Radians) 

0 

.  4 

Average 
Center 
of  Impact 

Standard 
Deviation 
of  Cl’s 

Average 
Center 
of  Impact 

Standard 
Deviation 
of  Cl's 

OE 

>22.48 

143.15 

-9.20 

129.36 

«, 

-17.02 

241.27 

-5.70 

164.29 

1 

0*3 

-35.99 

173.23 

-8.65 

136.83 

0E4 

-17.40 

445.22 

•  2.88 

232.92 

M 

-19.02 

141.06 

-3.26 

126.60 

SB 

-18.02 

231.10 

-10.02 

166.50 

20 

— 

-33.37 

170.03 

-11.72 

137.54 

wsm 

-11.87 

411.32 

9.83 

251.30 

Deflection 
■  200 meters 


1 

0E1 

oe2 

°e3 

0E4 

0  03 

0.09 

0.02 

-0.07 

3.00 

4.71 

3.28 

12.22 

8.60 

11.85 

11.25 

9.49 

3C.76 

62.45 

42.26 

99.04 

0E! 

■81 

18.07 

7.76 

0E2 

18.81 

13.09 

20 

°‘3 

JH 

18.08 

12.35 

44.98 

OE4 

0.06 

28.36 

9.32 

97.04 

*  The  observer  Standard  Deviations  are  defined  in  footnotes  on  Page  326 
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TAB LB  V 

Meant  and  Dispersions  of  Rang*  and  Deflection 
Center  of  Impacts  For  a  Target  of  Radius  25  Meters 


Range 

oy  •  25  meters 


Deflection 
Rd.-to-Rd. 
Std.  Dev. 

(o  ) 

X 

Observer* 

Standard 

deviation 

[o  to  ) 
v.e  bs  . 

0T  Ana 

Le  (Radians!  i 

0 

Average 
Center 
of  Impact 

Standard 
Deviation 
of  Cl’s 

Average 
Center 
of  Impact 

Standard 
Deviation 
of  Cl’s 

■■ 

— 

mm m 

30.34 

mgm 

34.60 

WM 

1 

149.51 

106.96 

64.86 

-4.69 

64.96 

wEm 

1 

319.45 

-9.39 

328.52 

mam 

-3.38 

29.02 

bw 

36.47 

-0.08 

138.41 

.  1 

122.83 

20 

n 

-6.45 

65.38 

-2.65 

67.98 

BH 

-3.31 

305.29 

9.17 

317.04 

Deflection 

c  ■ 

y 

25  meters 

mam 

mm 

2.97 

3.07 

15.68 

Wm-' 

1 

5.77 

0.00 

43.08 

1 

WM 

H 

3.67 

3.18 

26.69 

mm 

-0.02 

16.09 

0.83 

106.77 

mam 

0.19 

19.50 

1.78 

24.85 

mxsm 

22.41 

1.68 

54.55 

20 

■M 

BEfl 

20.65 

1.85 

33.73 

Hal 

BBS 

35.86 

0.83 

112.95 

*  The  observer  Stsndard  Deviations  are  defined  In  footnotes  on  Page  326 
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TABLE  V  (Cont'd) 

Meant  and  Dltperslont  of  Range  and  Deflection 
Center  of  Impacts  For  a  Target  of  Radius  25  Meters 


Range 

oy  •  50  meters 


OT  Annlo  (Radians) 


Deflection 

Observer* 

_ -  0  _ _ 

QMHHHHHf 

Rd  •  •  to*"Rd  • 

Standard 

Average 

Standard 

Standard 

Std.  Dev. 

Deviation 

Canter 

Deviation 

Center 

Deviation 

(8  ) 

X 

(\e  *  5xe> 

of  Impact 

of  Cl 'a 

of  lapact 

of  Cl's 

0E1 

-4.41 

46.01 

-2.60 

49.27 

OE« 

-3.71 

145.65 

3.71 

134.65 

1 

«, 

-9.54 

80.28 

-4.91 

80.62 

0E4 

2.05 

369.61 

5.59 

305 . 90 

-4.86 

45.81 

mm 

49.81 

20 

■ 

2.03 

141.42 

v. 

131.97 

-5.33 

81.49 

-a. 61 

72.97 

3.93 

314.51 

-0.13 

294.26 

Deflection 
0^  «  50  meters 


1 

0E: 

°e2 

°e3 

oe4 

-0.00 

-0.06 

-0.01 

0.34 

3.13 

6.07 

3.71 

18.51 

2.39 

0.28 

2.17 

1.39 

15.80 

55.27 

29.83 

107.87 

M 

-0.85 

19.61 

1.80 

25.31 

-0.14 

23.34 

0.32 

58.18 

20 

— 

0.09 

20.19 

2.52 

34.40 

In 

-0.13 

36.52 

3.60 

109.38 

*  The  observer  Standard  Deviations  are  defined  in  footnotes  on  Page  -*26 


TABLE  V  (Coru'd) 

Means  and  Dispersions  of  Range  and  Deflect  i'- 
Center  of  Impacts  For  a  Target  of  Radius  25  Meters 


Range 

0^  ■  100  meters 


Deflection 
Rd.-to-Rd. 
Std.  Dev. 

(ax> 

Observer* 

OT  Anele 

(Rr.dians) 

0 

■ 

■n 

■hni 

Standard 

Deviation 

(o  &  o  ) 

ve  xe 

Average 
Center 
of  Impact 

Standard 
Deviation 
of  Cl *s 

Average 
Center 
of  Impact 

Standard 
Devi ation 
of  Cl’s 

OE. 

1 

-14.92 

84.25 

-4.62 

80.55 

oe2 

-3.98 

185.85 

-0.13 

154.45 

1 

0E3 

-21.11 

117.37 

-6.51 

101.89 

oe4 

-0.29 

424.59 

6.52 

305.82 

0E1 

84.46 

82.69 

OE0 

388.44 

148.52 

20 

0E3 

111.65 

-7.40 

102.81 

oe4 

394.30 

13.71 

335.32 

Deflection 


*  The  observer  Standard  Deviations  are  defined  in  footnotes  on  Page  326 
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TABLE  V  (Cont'd) 

Means  and  Dispersions  of  Range  and  Deflection 
Center  of  Impacts  For  a  Target  ot  Radius  25  Meters 

Range 

0^  -  150  meters 


Deflection 
Rd.-to-Rd. 
Std .  Dev. 
(o  ) 

X 

Observer* 

0T  Ancle 

(Radians) 

0 

. 1 

Standard 

Deviation 

(o  6  o  ) 
ve  xe 

Ave  rr.gc 
Center 
of  Impact 

Standard 
Deviation 
of  Cl’s 

Average 
Center 
of  Impact 

Standard 
Deviation 
of  Cl’s 

OE 

-14.16 

117.10 

-5.88 

110.30 

-2.60 

218.09 

-3.12 

177.98 

1 

-25.00 

142.83 

-12.32 

135.53 

«* 

-8.78 

424.66 

11.48 

368.45 

OE 

-15.35 

113.26 

-7.44 

111.62 

«2 

-6.04 

215.40 

-3-63 

191.63 

20 

«, 

-18.78 

144.91 

-12.97 

134.99 

OE4 

-10.02 

404.01 

15.94 

(  405.14 

Deflection 
o  •  150  meters 

y 


1 

0E1 

oe2 

0E3 

0E4 

0.C5 

0.00 

-0.06 

0.21 

3.00 

7.84 

3.60 

18.21 

4.53 

4.73 

7.25 

0.83 

25.19 

66.53 

43.80 

120.54 

jmm 

0.28 

19.33 

3.44 

31.24 

Bn,- 

-0.37 

22.59 

4.96 

73.02 

20 

0.17 

19.98 

5.12 

47.59 

0.44 

34.81 

. 

0. 34 

125.48 

*  The  observer  Standard  Deviations  are  defined  in  footnotes  on  Page  326 


319 


TABLE  V  (Cont'd) 

Means  and  Dispersions  of  Range  and  Deflection 
Center  of  Impacts  For  a  Target  of  Radius  25  Meters 

Range 

a  "  200metcrs 

y 


Deflection 
Rd.-  to-Rd. 
Std.  Dev. 

<v 

Observer* 

Standard 

Deviation 

(*ve  &  V 

Of  Anale 

(Radians) 

0 

.4 

Average 
Center 
of  Impact 

Standard 
Deviation 
of  Cl's 

A\ :rage 

Cen  ter 
of  Impact 

Standard 
Deviation 
of  Cl's 

0El 

-18.14 

148.54 

-4.76 

139.40 

<*2 

-14.21 

247.62 

t rt 
00 

CM 

1 

214.57 

1 

OE 

-28.24 

178.58 

-12.45 

164.59 

«. 

-8.60 

471.61 

2.74 

397.00 

OE 

-13.14 

146.54 

-7.57 

141.25 

0E2 

-10.14 

246.22 

2.98 

217.09 

20 

« 

-26.63 

177.32 

-7.57 

168.67 

4.33 

453.49 

3.07 

395.03 

Deflection 
o  -  200  meters 

y 


1 

0E1 

oe2 

oe3 

°E4 

-0.04 

-0.01 

-0.03 

0.00 

3.00 

7.47 

3.59 

19.59 

4.74 

5.63 

6.40 

-0.20 

25.43 

73.69 

48.04 

131.96 

0E! 

-0.32 

19.03 

4.20 

20 

mam 

0.39 

22.66 

3.14 

75.80 

-0.19 

19.58 

4.48 

51.31 

19 

0.65 

35.86 

1.27 

134.61 

*  The  observer  Standard  Deviations  are  defined  in  footnotes  on  Page  326 
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TABU  VI 

Mean*  and  Dispersion*  of  Range  and  Deflection 
Canter  of  Impacts  For  a  Target  of  Radius  100  Meters 


Range 

0^  “  25  meters 


Deflection  1 
Rd.-to-Rd. 
Std.  Dev. 

<°x> 

Observer* 

Standard 

Deviation 

(o  4  o  ) 
ye  xe 

OT  Ancle 

(Radians) 

0 

.2 

Average  j 

Center 
of  Impact 

Standard 
Deviation 
of  Cl’s 

Average 
Center 
of  Impact 

Standard 
Deviation 
of  Cl’s 

0E1 

59.81 

-0.64 

75.95 

205.04 

17.76 

193.44 

1 

ob3 

-0.33 

129.27 

1.17 

115.27 

20.59 

432.14 

20.79 

462.34 

0E1 

-0.51 

59.28 

0.25 

72.41 

20 

0E2 

10.79 

214.46 

10.34 

192.52 

0E3 

2.84 

128.65 

-0.70 

118.71 

oe4 

3.33 

427.99 

19.17 

453.15 

Deflection 
o  ■  25  meters 

y 


1 

0E1 

oe2 

OE3  ; 

oe4 

0.04 

0.28 

-0.00 

0.24 

3.94 

16.34 

6.91 

31.07  1 

0.10 

-5.02 

-0.80 

-4.08 

26.26 

75.11 

43.50 

144.18 

0E. 

-0.35 

21.05 

0.20 

32.29 

0E2 

0.27 

26.65 

-3.74 

78.14 

20 

0B 

-0.26 

i 

22.06 

0.71 

47.30 

°*4 

1.57 

43.85 

-2.52 

139.04 

TABLE  VI  (Cont'd) 

Keans  and  Dispersions  of  Range  and  Deflection 
Cantor  of  Impacts  For  a  Target  of  Radius  100  Meters 


Range 

3^  ■  50  meters 


Deflect* on 
Rd.-to-Rd. 
Std.  Dev. 

<  x> 

Of  Annie 

(Radians) 

( 

Observer* 

Standard 

Deviation 

(  &  ) 
_ye  xe 

0 

.4 

Average 
Center  | 

of  Impact 

Standard 
Deviat ion 

c  f  or  a 

Average 
Center 
of  lmpaec 

Standard 
Deviation 
Of  Cl  *  8 

0E1 

69.83 

-0.39 

82.86 

«2 

206.22 

17.92 

204.53 

1 

131.12 

0.26 

125.24 

«. 

20.10 

435.05 

8.83 

413.07 

0E1 

0.04 

70.42 

-0.06 

84.46 

0E 

14.62 

211.28 

15.87 

199.47 

20 

- 

-0.12 

133.63 

0.05 

122.20 

12.03 

400.08 

13.99 

440.17 

Deflection 
o  *50  meters 

y 


1 

0E1 

°e2 

°e3 

0t4 

-0.00 

0.32 

-0.22 

-0.05 

3.88 

16.13 

7.14 

30.21 

-0.08 

-5.12 

0.53 

-1.29 

26.54 

77.96 

45 . 46 

135.40 

0E 

0.11 

20.99 

0.01 

32.95 

OP, 

-0.29 

26.  70 

-4.83 

79.53 

0E3 

0.14 

22.10 

0.27 

48.32 

0E4 

-0.31 

44.13 

-2.68 

143.17 

*  The  observer  Standard  Deviations  are  defined  in  footnotes  on  Page  3>26 
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TABLE  VI  (Cont’d) 

Means  and  Dispersions  of  Range  and  Deflection 
Center  of  Impacts  For  a  Target  of  Radius  100  Meters 


Range 

o  ”  100  meters 

y 


1 

Deflection  ] 

Observer* 

O’l  An  ale 

(Radians) 

* 

0 

] 

.4 

Rd.-to-Rd. 
Stdt  Dev. 

<°*> 

Standard 

Deviation 

(C\e  *  °‘xe) 

Average 
Center 
of  Impact 

Stundnrd 
Deviation 
of  Cl's 

Average 
Center 
of  Impact 

Standard 
Deviation 
of  Cl's 

OE 

-9.58 

101.55 

-7.15 

106.66 

4.75 

231.91 

9.61 

219.01 

1 

«3 

-10.68 

160.91 

-6.00 

148.55 

«! 

17.33 

470.25 

15.20 

449.81 

-8.34 

100.41 

-7.16 

105.14 

9.27 

239.88 

14.30 

228.55 

20 

B  • 

-11.43 

157.05 

-5.24 

147.40 

-1.75 

445.52 

5.20 

443.25 

Deflection 
o  “  100  meters 

y 


1 

CE1 

°e2 

0E3 

0E4 

0.05 

-0.07 

-0.00 

0.01 

3.88 

15.43 

7.02 

26.27 

2.98 

-1.68 

0.93 

-4.13 

29.44 

80.72 

49.97 

145.69 

OE 

-0.17 

20.48 

3.58 

35.01 

°E2 

0.77 

26.66 

-2.19 

83.33 

0.29 

21.75 

2.51 

53.71 

OE. 

_ h _ 

-0.25 

41.96 

-0.91 

oc 

L/1 

*The  observer  standard  Deviations  are  defined  in  footnotes  on  I'nj;  c  326 
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TABLE  VI  (Cont'd) 

Means  and  Dispersions  of  Range  and  Deflection 
Center  of  Impacts  For  a  Target  of  Radius  100  Meters 

Range 

0^  •  150  meters 


Deflection 
Rd.-to-Rd. 
Std.  Dev. 

Observer* 

Standard 

Deviation 

(°vc  4  °xe> 

OT  An  g 1 

;  (Radians) 

0 

. 

Average 
Center 
of  Impact 

Standard 
Deviation 
of  Cl's 

.■‘'•erage 
Center 
of  Impact 

Standard 
Deviation  \ 
of  Cl’s 

0E1 

-7.77 

132.62 

-6.61 

136.23 

8.70 

256.10 

12.93 

241.38 

1 

-9.37 

180.40 

-6.63 

171.4) 

“4 

13.78 

456.82 

13.01 

458.73 

| 

-6.93 

130.43 

-3.93 

135.69 

1.25 

250.75 

15.09 

243.93 

20 

H 

11.86 

176.42 

-7.77 

173.14 

Mm: 

10.98 

480.80 

18.54 

479.77 

'  1 

Deflection 
a  ■  ISO  meters 

y 


1 

0E1 

oe2 

0E3 

0E4 

0.03 

0.17 

0.02 

0.73 

3.84 

15.17 

5.35 

33.31 

30.68  | 

81.83 

1 

53.71  1 

1 

147.95 

mm 

D 

20.24 

1.27 

35.28 

■gym 

26.80 

-2.26 

86.86 

20 

mam 

-0.08 

21.80 

1.07 

58.02 

iKSfllllli 

0.56 

44.52 

-2.03 

152.15 

_  ,  .....  i 

*  The  observer  Standard  Deviations  are  defined  In  footnotes  on  Page  326 
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TABLE  VI  (Cont'd) 

Means  and  Dispersions  of  Range  and  Deflection 
Center  of  Impacts  For  a  Target  of  Radius  100  Meters 


Range 

"  200 meters 


0T  Angle 

(Radians) 

Deflection 
Rd.-to-Rd. 
Std.  Dev. 

(V 

Observer* 

Standard 

Deviation 

(o  So) 
ye  vc 

0 

■ 

mammm 

Average 
Center 
of  Impact 

Standard 
Dcviatio 
of  Cl's 

Average 
Center 
of  Impact 

Standard 
Deviation 
of  Cl’s 

0E1 

166.80 

3.40 

167.65 

0E, 

285.29 

10.84 

260.13 

1 

-11.25 

208. 85 

-8.22 

2- H.08 

0E4 

14.69 

528.87 

12.77 

509.48 

0R1 

-9.22 

161.78 

-1.75 

166.42 

oe2 

2.04 

279.94 

15.98 

267.05 

20 

oe3 

-8.76 

207.45 

-8.67 

201.25 

<*« 

17.54 

498.78 

4.12 

540.15 

Deflection 
o  *200  meters 

y 


1 

0E1 

°e2 

0E3 

OE, 

4 

-0.02 

0.13 

-0.02 

0.84 

3.43 

14.51 

6.13 

32.19 

1.91 

0.39 

2.67 

-3.65 

30.02 

87.71 

59.49 

154.20 

0Ei 

-0.13 

20.38 

0.87 

36.00 

OE, 

0.13 

26.71 

-3.96 

89.88 

20 

«, 

-0.67 

21.27 

1.91 

60.26 

0E4 

-0.69 

44.17 

-0.67 

156.92 

*  The  observer  Standard  Deviations  are  defined  in  footnotes  on  Page  326 
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FOOTNOTES 


F ~ 


*  Hie  observer  Errors  are  defined  below: 


OBjl 

o  ■ 

y« 

.2  (y) 

1 

0  » 

xe 

.1 

(x) 

0E2: 

V  " 

•2(4y) 

• 

» 

0  " 
xe 

.1 

<4x> 

OEj! 

" 

.5  (y) 

* 

> 

A 

0  ■ 
xe 

.2 

(x) 

0B. : 

4 

V  " 

•5(4y) 

» 

n  • 

xe 

.2 

(4x) 

where:  y  and  x  are  the  range  and  deflection  miss  distances  of  the 
impact  point  from  the  target  center,  respectively.  OE^  and 
OE„  are  used  when  the  OT  distance  is  less  than  1000  t<et.*rs 
while  OEj  and  OE^  are  used  ’•hen  the  OT  distance  is  between 
3000  and  4000  meters. 
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QUICK  REACTION  STUulf  OF  CALIBRATION  DRIFT 
IN  RADIAQiETER  1M-174Q 


Joseph  Weinstein 
Systecs/Cost  Analysis  Office 
US  Amy  Electronics  Co— end 
Fort  Monmouth,  New  Jersey 


1.  BACKGRQUHE 

My  purpose  Is  to  show  how  the  use  of  an  experlncnt.il  design: 

a.  Verified  an  allegation  that  a  problem  of  "drift**  In  calibration 
existed  In  ‘he  Radlacneter  IM-174 (). 

b.  Estinated  the  size  of  the  drift  (which  was  useful  In  recommending 
a  "quick  fix**  on  the  problen). 

c.  Satisfied  a  tins  constraint  given  to  the  co— odlty  manager 
for  responding  to  the  allegation. 

d.  Identified  the  causitlve  factor  for  corrective  action  attention. 

e.  Saved  engineering  test  samples,  facility  test  tine  and  time 
costs  of  skilled  personnel  of  the  order  of  90X  of  the  conven¬ 
tional  approach  that  would  have  bees  taken  without  the  "short 
fuse"  tine  constraint. 

The  urgency  for  resolution  of  the  problen  rested  oo  the  fact  that  the 
instrument  was  authorized  for  Issue  and  urns  In  short  supply.  Two 
production  lines  had  been  stopped  by  the  contracting  officer  when  the 
allegation  of  fault  was  made  and  he  was  assuming  penalty  payments  while 
waiting  for  engineering  guidance  for  termination,  engineering  cha.-gee  <r 
continuance  of  the  production. 

It  would  be  incorrect  to  think  that  our  engineer-managers  immedi¬ 
ately  recognized  that  they  had  a  problen  where  the  services  of  a  desigr 
statistician  were  needed.  Like  those  engineers  in  other  laboratories 
and  engineering  organizations  they  had  prepared  a  conventional  plan  of 
study.  It  involved  some  1300  instruments  which  would  be  tested  under 
"crash"  programs  and  a  minimum  time  six  nontfcs  would  txr  useo  to 
obtain  the  data  necessary  for  a  roccoBendaVion,  Since  the  penalty 
costs  were  In  the  neighborhood  of  $1000  or  $1500  a  day  on  each  contract 
the  commodity  manager  turned  to  the  last  source  of  help  -  the 
statistician.  We  held  several  short  orientation  meetings  and  studied 
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background  documents,  test  reports  and  some  recent  test  data  from  the 
Lexington  Army  Depot  on  12  instruments  which  had  concluded  that  the 
'’drift"  problem  was  real  and  primarily  found  in  instruments  from  Depot 
Stock  (IM  108/U)  which  had  been  modified  and  "improved"  at  the  depot 
and  had  been  recalibrated  shortly  before  these  tests  were  made.  This 
report  also  thought  that  the  causes  were  to  he  found  with  the  source 
of  the  instrument,  the  length  of  time  it  had  been  stored  and  the  "aging" 
cycle  that  had  been  experienced  by  the  #5836  electrometer  tube  during 
its  manufacturing  cycle.  ("Expert"  tube  engineers  were  so  sure  of 
this  cause  that  Raytheon  Corp.  offered  to  speedily  redesign  the  tube 
itself  If  we  at  WT'H  would  assume  the  costs.) 

A  careful  review  of  the  data  on  these  12  instruments  in  conjunction 
with  a  physical  examination  showed  that  the  readings  were  made  on  a 
logarithmic  scale  (covering  2.5  decades  of  radiation  intensity  in 
2.25  inches  of  arc.)  -  strongly  suggested  that  the  "drift"  could  easily 
be  associated  with  biased  reading  error. 

Since  the  readings  were  made  at  a  radiation  level  where  large 
reading  errors  could  occur  in  drift-free  instruments  and  since  a  "run" 
of  6  readings,  each  smaller  than  its  predecessor,  had  a  small,  but  not 
infinitesimal,  probability  of  occurrence  with  drift-free  instruments, 
there  was  sufficient  basis  for  deferring  judgment  until  additional  infor¬ 
mation  from  a  well-controlled  test  urns  available,  if  it  could  be 
acquired  quickly.  It  was  agreed  that  a  verification  experiment  ought 
to  be  done  before  funding  the  redesign  of  the  tube.  If  possible,  it  was 
also  desirable  to  estlmete  the  influence  of  tbe  source  of  manufacture, 
the  aging  cycle  used  In  the  5886  tube,  the  time  spent  in  storage  prior 
to  test,  and  (as  an  after  thought)  a  simulation  of  the  experience  of 
(tolar  heating  in  storage  depots  by  a  temperature  cycling  of  4  hours  at 
160° F  in  every  24  hours  of  storage. 

2.  THE  TEST  FLAK; 

The  test  plan  developed  is  given  below.  It  uses  128  instruments 
instead  of  the  1500  considered  and  the  entire  program  of  testing  is 
completed  in  8  weeks  with  a  report  submitted  by  tbe  10th  week  instead 
of  the  26  veexs  of  testing  proposed.  It  considers  more  factors, 
estimating  their  Joint  effects,  and  gives  reasonably  precise  estf nates 
of  each  factor's  influence  on  the  drift  observation.  All  of  these  out¬ 
puts  being  obtained  with  roughly  iOX  of  the  effort  proposed  by  the 
engineering  group  were  completed  and  submitted  to  the  program  officer 
by  the  10th  week  after  initiation. 

* 

These  were  to  be  from  2  producing  sources  however  a  third  source  was 
developed  and  an  additional  64  instruaents  were  made  available. 
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A  detailed  test  plan  had  to  be  prepared  for  the  remote  testing 
facility  (at  the  Lexington  (Ky)  Signal  Depot).  The  plan  covered  5 
pages  of  single  spaced  information  with  step-by-step  procedures  for 
selection,  pre-conditioning  initial  calibration,  and  the  test  envir¬ 
onment  to  be  assigned  to  each  sample  unit.  Since  the  facilities  for 
test  performance  were  limited  (and  the  time  for  completion  as  well) 
it  was  necessary  to  consider  tight  time  scheduling  for  each  test  sample 
unit  and  do  partial  data  analysis  while  tests  were  continuing.  All 
details  relating  to  these  considerations  are  shown  in  plates  1-9 
inclusive. 


The  factors  and  levels 

considered 

were 

• 

Storage  time 

1 

(1) 

8 

weeks 

Producing  source 

1. 

■» 

— • 

3 

(8  units  from  each  source 
per  week) 

Aging  schedule  of 

24 

<24> 

96 

hours  of  additional  aging 

Temperature  cycling  room  temperatures  vs  4  hours  at  160-F 

Han  of  the  Data  Analysis. 

The  measure  of  "instability*  for  the  test  sample  unit  was  the 
absolute  value  of  the  changt  in  reading  of  the  instrument  between  the 
calibration  value  at  the  start  of  the  experiment  and  the  measurement 
at  Che  end  of  the  .'.cheduled  test  time.  After  this  procedure  had 
established  the  existence  of  i  "drift",  the  arithmetic  value  of  the 
change  could  be  examined  for  the  direction  and  magnitude  of  "drift.” 
These  measures  of  "drift”  were  made  for  three  level*  of  the  radiation 
400  r/hr,  200  r/hr,  and  50  r/hr.  Results  from  the  experiment  became 
available  weekly  providing  the  analyst  with  16  (24)  randomly  chosen 
Instruments  which  had  been  exercised  by  the  test  plan  ’V"  weeks 
(w  ■  l  to  3).  Of  these  8  instruments  were  from  each  producing  source; 

2  of  each  8  had  electrometer  tubes  at  each  of  the  tube  aging  schedules 
and  4  of  each  8  (1  at  each  of  the  4  tube  aging  schedules)  were 
given  the  temperature  cycle  exposure.  These  were  subjected  to  analysis 
of  variance  techniques  for  the  estimation  of  each  factor’s  influence 
on  the  drift  phenomenon. 

3.  TEST  RESULTS  AND  ANALYSIS. 

(a)  Pre-test  Calibration  data. 

Prior  to  exposure  of  thn  test  units  to  t>e  schedule  of  test 
conditions  developed.  It  was  necessary  to  verify  their  compliance  with 
specified  performance  requirements.  This  is  indicat'd  graphically  by 
plates  10,  11,  and  12  each  for  a  different  radiation  exposure  level  and 
each  depicting  the  distribution  of  the  samples  from  the  three  producing 
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source*.  With  minor  exceptions  the  ter'-  unite  comply  with  specific 
velues.  It  Is  also  observed  that  whereat,  the  200  r/hr  data  are  closely 
grouped  and  symmetric  about  the  design  value  the  400  r/hr  data  and  the 
SO  r/hr  data  exhibit  broader  spreads  and  asymmetry.  The  depot  test 
units  suggest  the  possibility  of  design  differences  from  the  other  two 
producing  sources  since  the  direction  of  skewnees  for  these  units  is 
opposite  to  that  of  the  other  producers. 

(b)  Catastrophic  failures. 

An  unexpected  but  quite  important  result  was  the  observation 
that  a  large  proportion  of  the  inatruaents  failed  to  perform  after 
exposure  to  the  test  environment.  The  weekly  data  seemed  to  be  inde¬ 
pendent  of  time  but  heavily  dependent  on  the  temperature  cycling 
exposure.  Summarised  data  la  given  in  Table  1. 

TABLE  I:  Catastrophic  Failures  (38  Tested  at  aach 
_ _ condition): _ 


SOURCE  OP 

TEMPERATURE 

TEST  UNIT 

ROOM 

4  HRS 

at  leo'V  Daily 

Depot 

0 

13 

LFfcC 

a 

13 

Victory 

i 

10 

Psrcmnt 

3  •  .031 

38  » 

.393 

Failure 

TO 

TO 

(c)  Temperature  Cycling  end  Time. 

Plate  13  illustrate*  the  strong  effect  of  the  temperature 
exposure  over  the  8  vaeks  of  costing.  This  effect  is  independent  of 
tne  radiation  lavel  and  seams  to  be  dieappearing  at  the  8th  week  of 
expoeure.  Unfortunately  the  cestlng  was  so  tightly  scheduled  and 
the  need  for  decision  about  the  drift  problem  limited  the  opportunity 
to  pursue  this  conjecture  to  a  decision.  However,  the  Importance  of 
temperature  cycling  to  change  in  calibration  and  as  a  cause  of  cata¬ 
strophic  failures  is  unequivocal.  The  design  also  protected  against 
bias  due  to  scale  shifts  with  tima  which  appear  sinusoidal. 
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(d)  Production  Sour eta  and  Tima. 

Plat*  14  UluitritM  th*  relatively  random  influence  of  the 
producing  source  at  a  cause  of  change  from  calibration  values. 

(•)  Aging  Schedule  and  Time. 

Plata  IS  shows  hov  drift  with  time  was  Influenced  by  the  prior 
aging  of  tha  3886  electrometer  tube  bafora  incorporation  in  the  teat 
sample.  Here  again  we  find  relative  shifting  of  position  of  the  symbols 
associated  with  prior  tube  aging  so  that  no  indication  of  a  tuba  effect 
appaara  to  exist, 

(f)  Calibration  "Drift"  with  Time  by  Producing  Source  and 

Radiation  Laval. 

The  change  between  pre-test  calibration  and  post  test  Instrument 
reading  waa  regressed  on  the  time  axis  for  data  derived  from  the  8 
weeks  of  testing.  This  was  done  for  each  level  of  radiation  exposure 
and  producing  source  with  separate  regression  lines  for  the  instru¬ 
ments  held  at  room  tamperature  and  for  those  having  be^n  temperature 
cycled.  The  results  are  given  in  plates  16,  17.  &  18  for  a  400  r/hr 
radiation  lavel  and  in  plataa  19.  20,  &  21  for  200. r/hr  radiation  level, 
A  Hating  of  the  regression  coefficients  in  the  Table  II  below  shows 
positive  valued  coefficients  for  time  and  negative  constants  (initial 
values)  throughout,  with  the  exception  of  the  Victory  room  temperature 
data  at  400  r/hr. 


4.  SUMMARY I 

The  graphic  studies  of  the  averaged  absolute  changes  in  instrument 
readings  with  time  at  3  points  on  the  radiation  scale  (400,  200  and  SO 
rads/hour)  are  presented  as: 


Plate  13, 
Plate  14, 
Plate  15, 


IM-174  Study  of  Temperature  Cycling)  These  figures 


IM-174  Study  of  Source  Differences  )  ®re  kased  on 

Depot  units 

IM-174  Study  of  Electrometer  Tube  )  and  Landers, 


Aging 


Frary  &  Clark 
units  only. 
However,  they 
are  believed  to 
apply  as  well  to 


Victory  units. 
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From  chege  plates  -  13,  14,  13,  It  is  quite  apparent,  from  the 
veekly  rearrangement  of  the  order  of  the  averaged  absolute  changes 
for  different  amounts  of  aging  (Plata  15)  and  for  sources  (Plate  14), 
that  there  Is  no  consistent  difference  In  performance  of  the  IM-174() 
Instruments  for  these  factors.  From  Plate  13,  however,  it  la  quite 
apparent  that  temperature  cycling  plays  a  very  important  role  In  changes 
2rom  original  calibration  values.  These  appraisals  are  Independently 
confirmed  by  Analyses  of  Variance  performed  on  each  group  of  16  reports 
which  are  available  iu  the  files  of  the  undersigned  but  are  not  sub¬ 
mitted  as  part  of  this  report.  Having  established  that  a  well  defined 
effect  was  exhibited  between  instruments  temperature  cycled  and  those 
held  at  room  temperature  over  the  8  week  study  the  nature  of  this 
effect  was  studied  more  exactly  by  fitting  trend  lines  to  the  observed 
changes  including  the  sign  of  the  change.  This  is  showjj  in  Plates  16, 

17,  18,  and  19,  20,  21.  It  la  apparent  that  a  positive  time  drift 
exists.  The  validity  of  these  trend  coefficients  -  in  terms  of  ade¬ 
quately  describing  the  effect  of  the  assumption  that  a  linear  drift 
with  time  existed  In  the  data  -  was  ter  ted  by  performing  a  variance 
analysis  on  the  fitted  curves.  The  results  imply  a  high  degree  of  veri¬ 
fication  of  the  assumed  linear  trend  In  the  case  cf  units  stored  at 
room  temperature  with  residual  (unexplained)  variances  which  appear  to 
be  reasonable  as  estimates  of  the  instrument  reading  error.  However,  In 
Che  case  of  units  temperature  cycled  during  storage,  such  verification 
Is  somewhAt  clouded  by  the  loss  of  a  large  proportion  of  the  Instruments 
and  by  the  potential  of  the  temperature  cycling  to  have  stimulated 
several  different  modes  of  change  (or  failure)  not  present  in  the  rocm 
temperature  environments.  In  any  event  there  le  a  considerable  Increase 
In  residual  variation  about  the  trend  lines  fitted  to  such  data  and 
suggesting  a  reading  error  of  the  order  2  to  3  times  the  values  obtained 
in  the  room  temperature  fittings.  These  results  are  tabulated 
In  Table  II. 

5.  RECOMMENDATIONS  AND  CONCLUSIONS: 

The  alleged  "drift"  of  calibration  with  storage  time  Is  confirmed 
by  this  experiment.  However  Its  effect  is  so  small  relative  to  Influence 
of  temperature  cycling  and  possibly  poor  manufacturing  control  as  evi¬ 
denced  by  Table  I,  that  no  further  modification  effort  in  this  area  is 
recommended  until  the  gross  effects  to  be  reported  In  the  ongoing  engi¬ 
neering  study  by  Atomics  Branch,  USAELRDl  are  corrected  and  controlled. 
The  utility  of  -the  present  Instrument  In  field  use  can  be  measurably 
improved  by:  (1)  scheduling  a  3-4  month  recallbration  cycle  Instead  of 
the  present  6  month  cycle.  This  will  keep  the  Instruments  within  their 
specified  tolerances,  (2)  maintain  them  In  storage  conditions  which 
will  avoid  the  thermal  cycling  exposure  of  the  test  specification  until 
modlf lcatlons  proposed  by  the  engineering  report  can  be  made. 

The  Lexington  Army  Depot  data  on  12  Instruments  tested  in  the  spring  of 
1963  and  reported  In  May  1963  correspondence  to  USAD1SA  Indicated  a 
negative  drift. 
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TABU  H:  CHANGE  AS  A  FUNCTION  OF  TIMS,  TYPE  OF  STORAGE, 
AMD  LEVEL  OF  RADIATION 


Type  of 

Source  of 
Units 

Fittad  Trand 

Lina  Equation 
Change  in  (r/hr) 

Ara  Slopa 

Coaff Iolanta 

Statistically 

Significant? 

Eatiaata 

of  Reading 
Error  (r/hr) 

loot 

Teap. 

400  4/hr 

Landers, 

Fr ary  A  Clark 

5.21  -  20.70 

Yaa 

17 

Depot 

3.251  -  4.32 

Yaa 

15 

Ylctory 

7.501  +  5.21 

Yaa 

22 

200  4/hr 

Landers, 

Frary  4  Clark 

1.241  -  5.21 

Yaa 

7 

Dapot 

1.321  -  2.28 

Yaa 

6 

Victory 

2.481  -  2.28 

Yaa 

8 

Type  of 
8torage 

Source  of 
Units 

Fittad  Trand 

Lina  Equation 
Change  in  (r/hr) 

Ara  Slopa 
Coefficients 
Statistically 
Significant? 

Eatiaata 
of  Reading 
Error  (r/hr) 

Teap 
Cycling 
at  160°F 

400  r/hr 

Landers, 

Frary  A  Clark 

4.281  -  58.26 

No 

22 

Dapot 

6.141  -  55.23 

Marginal 

36 

Victory 

5.181  -  77.38 

Marginal 

30 

200  r/hr 

Landers , 

Frary  A  Clark 

4.341  -  44.13 

Yes 

12 

Dapot 

3.001  -  33.03 

No 

22 

Victory 

1.711  -  34.00 

No 

13 
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As  soon  as  one  sub-group  is  filled  by  the  randomization  scheme  used,  no 
further  assignments  are  to  be  made  to  that  sub-group  so  that  we  will  finally 
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Identification  Marking.  Assign  a  four -digit  code  number  to  each  unit 
The  coding  procedure  will  be  developed  as  we  go  through  the  preconditioning 
procedure.  At  this  point  we  may  use  a  oxxx  to  identify  Landers,  Frary  & 


RECHECK  SCHEDULE  BY  WEEKS  OF  STORAGE 
Storage  Codes:  Cell  items  are  run  #‘s  within  blocks  on  calibration  test  design. 
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IM-174  TEST  DESIGN:  CALIBRATION 


IM-174  TEST  DESIGN:  CALIBRATION  (Continued! 
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I M -174  STUDY  OF  SOURCE  DIFFERENCES 
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IM-174 :  400  r/hr  DATA 
Lf  I C  UNITS 
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RELIABILITY  TESTING  OP  WEAPON  SYSTEMS 
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Ronald  L.  Raclcot  i 

Banat  R&B  Laboracorlaa 
Watervltet  Araanal 
Watervllet,  New  York 

1*  Introduction t  * 

Ttaa  basic  problem  which  I  would  Ilka  to  offer  (or  dlacuaalon  and  * 

commence  stems  from  *7  personal  involvement  la  tha  preparation  of  Reliability/ 
Maintainability  Prograa  and  Teat  Plana  for  a  particular  cloaa  support  waapon 
ayataa.  One  of  tha  high  priority  requirements  for  thla  waapon  system,  aa 
wall  aa  othar  proposed  Army  Materiel,  la  incraaaad  reliability.  Quantified 
reliability  requirement*  have  consequently  bean  Included  In  the  QKR,  Syataa 
Specifications  and  System  Development  Plans  for  thla  weapon  system.  This  will, 
of  course,  necessitate  the  adequate  testing  and  assaaumant  of  the  achieved 
reliability  of  tha  waapon  system  and  its  components  at  various  stages  in 
Its  development  cycla. 

In  the  preparation  of  tha  reliability  test  plana,  a  number  of  difficulties 
ware  encountered,  particularly  In  tha  methods  to  be  uaad  for  establishing 
confidence  Intervals  on  the  reliebllity  of  components  for  a  special  class  of 
idealised  physical  situations.  Whet  I  seek  here.  then,  are  consents  and 
possible  practical  approaches  for  resolving  these  apparent  difficulties. 

2.  Definition  of  Reliability! 

I  would  first  Ilka  to  define,  as  clearly  as  I  can,  one  of  the  basic 
reliability  Indices  which  Is  specified  in  the  weapon  system  requirements, 
that  la,  the  "mission"  or  Interval  reliability  under  the  assumption  of  "ideal 
repair"  (Ref.  1).  A  mission  hare  Is  defined  In  terms  of  a  given  number  of 
rounds  and  miles  which  Is  a  small  fraction  of  the  total  expected  life  of  the 
weapon  system.  Reliability  Is  defined  then  as  the  probability  that  the  waapon 
ayataa  will  perform  Ite  Intended  function  for  the  specified  interval  (mission) 
under  atated  conditions  which  are  given  in  the  mission  profile  for  the  weapon 
ayetem.  The  assumption  la  made  that  at  the  beginning  of  a  mission,  ths 
weapon  system  le  In  en  operable  state. 

Ths  term  "Ideal  repair"  Just  mesne  that  given  s  failure  of  the  weapon 
system.  It  is  subsequently  repaired  or  renewed  with  any  part  or  component 
replaced  or  repaired  being  put  in  "as  new"  condition.  The  weapon  system 
la  then  available  for  further  combat  use.  Between  missions,  preventive 
maintenance  actions  can  be  performed  which  include  the  replacement  of  parts 
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or  components  which  ars  worn  and  ara  approaching  a  high  failure  rate 
condition  (preventive  maintenance  parts  replacement) . 

1  emphasise  the  above  points  to  make  It  clear  that  it  is  not  the 
reliability  for  first  mission  or  the  conditional  reliability  for  first 
failure  which  is  specified  here  but  rather  the  reliability  for  an  Interval 
of  time  at  any  point  during  the  weapon  life  at  which  time  che  weapon  system 
could  have  previously  experienced  one  or  more  failures.  In  general,  this 
reliability  is  transient  with  the  specifications  consequently  being  given 
as  an  average  reliability  over  the  entire  life  of  the  weapon  system, 

3,  Reliability  Theory; 

It  might  do  well  at  this  point  to  introduce  summarily  without  proof, 
soma  of  the  mathematics  Involved  in  computing  the  mission  reliability 
assuming  that  all  required  parameters  are  known  exactly.  To  start  with, 
consider  only  a  single  part  or  component.  Theoretically,  if  we  know  the 
reliabilities  of  the  individual  components,  the  eystem  reliability  can  be 
computed  using  an  appropriate  reliability  model  for  the  system.  Let  f(t) 
be  the  probability  density  distribution  of  times  to  first  failure.  We 
could  write  f(t)  as  f(t;6)  where  6  in  this  case  represents  one  or  more 
population  parameters  which  completely  define  the  density  distribution  f. 


The  initial  problem  usually  considered  in  elementary  reliability  theory 
is  the  so-called  "first  failure"  reliability  problem.  In  this  case 


R(t) 


Probability  tvat  a  given  component  will  not  fail  in 

time 

(O.t); 

1  -  F(t); 


f(t)dt. 


(1) 


The  "conditional"  interval  reliability  for  an  interval  (t,  t  +  At)  ie  given 
as 


R(t,  At) 


-  Probability  that  a  given  component  will  not  fail 
in  an  interval  (t,t  +  At)  given  that  the  component 
has  not  failed  up  to  time  t. 


R-C*+At2 

R(t) 


(2) 


It  is  useful  at  this  point  to  define  the  failure  rate  or  force  of  mortality, 
\(t)  (Ref.  1)! 
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X(t)dt 


Probability  that  a  componant  will  fall  In  tha 
incremental  time  intarval  (t,t+dt).  (3) 


! 


'  tit 


Mt  rka  firat  fallura  rallablllty  problem,  X (t)  la  of  tan  called  tha 
"ooalittoaal"  fallura  rata  whara  X(t)dt  la  tha  conditional  probability  of 
fallura  given  that  tha  coaponant  haa  not  fallad  up  to  tlaa  t.  Aa  will  bacoma 
evident  shortly,  howavar,  tha  tara  conditional  falura  rata  la  not  appro¬ 
priate  for  tha  ldaal  repair  caaa  alnca  falluraa  and  replacaaanta  prior  to 
tlaa  t  are  allowed.  In  teraa  of  tha  conditional  fallura  rellabllltlea  pra- 
vloualy  given  aa  equations  (1)  and  (2)  can  be  written  aa 


R(t) 


R't.At) 


whara  A(t)  » 


t 

“  axp  (-  f  X(t)dt) 
o 

t+At 

-  axp  (-  |  X  (t)dt ) 
t 

f(t)/(l  -  F(t)) 
f(t)/R(t) 


(4) 


Cone  id  tr  next  the  caaa  of  Ideal  repair.  Clearly  thla  caae  represents 
a  renewal  proceaa  where  a  part  or  component  la  renewed  after  each  failure. 

In  thla  lnatanca,  the  probability  distribution  of  times  to  the  1st,  2nd,... .nth 
failure  for  n  arbitrarily  large  are  required  to  eetabliah  the  failure  rate 
A(t)  at  any  given  time  t.  Let  f  (t)  be  the  probability  density  distribution 
of  the  time  to  the  Jth  failure  ^where  f ^ (t >  “  f(t),  the  fundamental  denelty 
of  time  to  first  failure.  From  renewal  theory,  the  following  expression 
relates  f  (t )  to  the  fundamental  first  failure  frequency  f(t)  (Ref.  1, 
page  143)^: 

UfjU)}  -  [L{f(t)}]J  (5) 

where  L  {•}  «■  Laplace  or  Fourier  transform. 

Also,  tha  failure  rate  X(t)  in  this  case  can  be  written  down  as  follows 
(Ref.  1,  page  143): 

X(t)  -  l  t  At)  (6) 

J-l  J 

This  Is,  of  course,  the  unconditional  failure  rate. 
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Taking  the  transform  of  aquation  (()  and  using  (5)  finally  glvaa  tha 
following  aquation  ralatlng  X (t)  to  tha  fundamental  flrat  tallura  fraquancy 
f(t)i 


LO(t»  •  ^tfeTT  <’> 

Thaoratlcally  than,  glvan  f(t),  wa  can  compute  X(t).  Inttrvel  or  mission 
ral lability  In  thla  lnatanca  la  chan  glvan  aa 

C+At 

R(t,At)  -  exp{-  f  X(t)dt>  (8) 


A  problaa  of  conaldarable  intaraat  In  reliability  theory  Involvea  parts 
or  components  which  exhibit  wear-out  phenomena  cauaed  by  such  mechanisms 
as  fatigue,  corrosion,  erosion,  abrasion  and  other  similar  mechanisms.  For 
this  case,  tha  conditional  failure  rate  for  the  first  failure  problem  la  an 
Increasing  function  of  time.  In  this  instance,  It  Is  possible  to  Increase 
Interval  or  mission  reliability  by  replacing  components  between  missions 
before  they  enter  the  hlgh-failure  rate,  vaer-out  regions  of  their  lives. 

Let  us  assume  that  a  given  part  la  to  be  replaced  after  it  has  been  in 
operation  for  a  time  period  T,  aaouming  It  does  not  fall  prior  to  this  time. 
A  replacement  In  thla  case  la  not  considered  a  failure.  The  fundamental 
frequency  distribution  of  times  to  first  failure  Is  now  changed  but  can  be 
determined  from  the  underlying  distribution  of  first  failure  times  without 
replacement,  f(t),  using  the  expression  (Ref.  1,  page  153) i 


fD(t) 

-  1 

P 

J-o 

where 

fm('} 

T 

0 

F(T) 

•  1 

11  -  F(T)]J  f  (t  - 

B 

for  0  s 
0,  otherwise 

f(t)dtj 


o 


JT) 

<  T; 


(9) 


"  scheduled  replacement  time. 


Figure  1  le  a  typical  plot  of  f  (t). 

P 
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*p(t> 


mmi 


For  the  combined  case  of  Ideal  repair  and  preventive  maintenance  parte 
replacement,  the  aeme  expreealona  as  were  previously  presented  for  Just  the 
Ideel  repair  case  apply  here  with  the  density  f(t)  now  being  replaced  with 
the  preventive  parts  replacement  density  f  (t).  We  now  have  at  least  the 
theoretical  means  of  computing  interval  (mission)  reliability  as  a  function 
of  time  for  the  Ideal  repair  aid  parts  replacement  cases  assuming  that  we 
know  exactly  the  underlying  first  failure  distribution  f(t). 

A.  Reliability  Testing  Problems i 

a.  Determination  of  the  Confidenced  Reliability  of  Components  from 
Failure  Data 


In  the  Initial  stages  of  weapon  system  development,  tests  are  to  be 
conducted  on  prototype  systems.  The  general  philosophy  during  these  early 
tests  will  be  to  use  as  little  preventive  maintenance  as  possible  In  order 
to  purposely  generate  failures.  In  addition  to  the  prototype  tests, 
laboratory  simulation  tests  will  be  conducted  on  a  number  of  critical 
weapon  system  components,  again  with  the  intent  of  generating  failures.  The 
general  purpose  of  this  type  cf  approach  Is  to  attempt  to  pinpoint  as  early 
as  possible  significant  design  problems  as  well  as  to  generate  failure  data 
to  establish  preventive  maintenance  schedules  Including  parts  replacement. 
One  type  of  data  collected  during  these  teste  will  be  the  failure  times  for 
each  part  which  exhibited  failures.  It  le  desirable  at  the  end  of  these 
Initial  tests  to  assess  the  achieved  reliability  of  the  weapon  system 
Including  a  projected  parts  replacement  policy.  We  currently  plan  to  do 
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this  on  a  component-peccomponent  basis  and  then  to  compute  system  reliability 
using  an  appropriate  reliability  model  for  the  system:  for  example,  for 
the  series  reliability  model,  svstem  reliability  Rg(t)  1st  determined  from 

n 

R-(t)  -  R  R.(t)  (10) 

5  i-1  1 

where  n  -  total  number  of  components 
R^(t)  •  reliabiliry  of  the  ith  component 

In  general 

RS(C)  -  h(R  (t));  (11) 

*  given  function  of  the  individual  component  reliabilities. 

The  usual  assumption  of  Independence  of  failures  between  different  com¬ 
ponents  is  made  in  arriving  at  equations  (10)  and  (11). 

The  specific  problem  here  Is  how  to  compute  the  confidence  lncerval 
for  the  ideal  repair  mission  reliability  for  individual  components  given  the 
failure  times  (l.e.,  a  sample  from  the  population  f(t))  and,  if  applicable, 
the  replacement  time  V,  Assume  that  the  form  of  the  underlying  first 
failure  distribution  is  known;  e.g.,  Weibull,  Gamma,  Normal,  exponential, 
etc.  For  the  general  case,  the  mission  reliability  is  given  as 

R(t,At;6)  -  g(f (t,e))  (12) 

where  g(<)  is  a  known  function  of  f(t;v),  the  first  failure  density.  For 
the  preventive  parts  replacement  case,  only  the  lower  tall  of  f ( t ; 0 >  enters 
into  the  reliability  computation,  e  represents  one  or  more  unknown  popu¬ 
lation  parameters. 

Special  Cases: 

(1)  Exponential  or  Constant  Failure  Rate  Cau» 

In  this  instance,  failures  are  purely  random  and  do  not  depend 
in  any  way  or.  the  previous  history  of  failures.  There  is  no  advantage  to 
be  gained  in  using  u  parts  replacement  schedule  and  determination  of 
confidenced  mission  reliability  ir.  this  case  is  straightforward  even  for 
the  ideal  repair  case. 

(2)  Lew  MTBF  (mean  time  between  failures)  Case, 

Fur  system  times  of  approximately  3  to  U  times  the  MTBF  or  greater, 
the  failure  rate  X  (t)  approaches  a  constant  value  for  the  Ideal  repair  case 
and  Is  equal  to  1/MlflF.  For  the  preventive  parts  replacement  case,  the  MTBF 
must  be  based  on  ‘he  density  f  (t)  rather  chan  f(t).  Mission  reliability 
then  for  ler  e  system  draes  ^car.  be  computed  from  the  relation 
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R(t , At )  -  exp  (-At/MTBF) 


(13) 


Whet  le  required  in  this  lnstence  then  Is  e  confidence  interval  on  the  MTBF 
which  in  turn  depends  on  the  underlying  first  failure  distribution  (either 
f(t)  of  f_(t)).  This  seems  to  be  a  particularly  bothersome  problem  for 
the  preventive  parts  replacement  case  where  only  the  lower  tails  of  the 
original  first  failure  distribution  f(c)  enter  into  the  determination  of  f  (t). 
It  should  be  emphasised  hare  that  even  though  the  failure  rate  is  constant; 
the  flow  of  failures  in  the  time  direction  is  note  purely  random  process; 
that  is,  the  Interarrival  times  are  not  exponentially  distributed, 

b.  Determination  of  the  Confidence  Interval  for  Total  System  Mission 
Reliability  from  Individual  Component  Tests 

Assume  that  the  system  reliability  can  be  computed  from  the  component 
reliabilities  using  equation  (11),  l.e. 

Rg(t)  -  h(R1(t)),  i  -  . . . 

where  R^(c)  «  R^(t;6^) 

9^  •  population  parameters  for  the  ith  component 
n  ■*  number  of  components 

h(*)  ■  known  function  of  the  component  reliabilities. 

Given  the  results  of  individual  component  failure  times,  the  problem  here 
is  how  to  compute  the  confidence  Interval  for  the  system  mission  reliability, 

c ,  Determination  of  the  Confidence  Interval  for  System  Mission 
Reliability  from  Total  System  Tests 

In  the  latter  stages  of  weapon  system  development,  advanced  production 
engineering  prototype  systems  wi.l  be  subjected  to  full  scale  life  tests. 
Although  failure  times  for  each  component  comprising  the  weapon  system  will 
be  recorded  in  the  course  of  the  tests,  it  may  be  advantageous  in  computing 
confidence!  mission  reliability  to  treat  the  system  as  if  it  were  a  single 
unit.  The  eyetem  failures  in  this  case  would  then  be  a  composite  or  overlay 
of  all  the  mission  stopping  component  failures  that  are  experienced  during 
cne  teat.  Under  certain  idealized  conditions,  it  may  be  possible  to  treat 
such  system  failures  as  a  simple  Poisson  flow,  l.e.  purely  random  occurrence 
of  failure  (ref.  1,  pages  129-135).  In  this  instance,  confidenced  miss. on 
reliability  can  be.  readily  determined. 


Based  on  prior  experience,  however,  the  frequency  of  aystea  failures 
generally  increases  toward  the  latter  stages  of  system  life.  This  situation 
is  common,  for  example,  in  the  case  of  the  auto.  After  a  large  number  of 
miles  have  be on  accumulated,  failures  begin  to  occur  at  an  increasing  rate 
usually  through  the  failure  of  components  not  previously  failed  (e.g, 
radiator,  transmission,  motor  parts,  ate.).  In  this  case,  the  flow  of 
failure  is  no  longer  simple  Poisson.  The  basic  problem  then  is  how  to 
compute  confidenced  mission  reliability  for  this  particular  situation. 

REFQIENCES: 

(1)  Erich  Pieruschka,  "Principles  of  Reliability,"  Prentice-Hall,  Inc., 
Englewood  Cliffs,  New  Jersey,  1963. 

(2)  B.  V.  Gnedenko,  Yu.  K.  Belyayev,  and  A.  D.  Solovyev,  "Mathematical 
Methods  of  Reliability  Theory,"  Academic  Press,  New  York,  1969. 
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COMMENTS  BY  MURRAY  A.  GEISLER 
on 

"Reliability  Testing  of  Weapon  Systems" 
by 

Ronald  L.  Raclcot 


1.  Prom  previous  day's  discussion  on  field  testing,  we  understand  that 
the  reliability  of  a  weapon  system  Is  much  more  complex  than  getting  the 
reliability  of  individual  components.  Weapons  are  used  In  a  much  more 
complex  environment  than  is  represented  by  the  serial  product  of  probabilities. 

2.  A  more  meaningful  measure  of  the  capability  of  a  weapon  system  would  be 
operational  readiness  more  than  Just  reliability  alone,  since  operational 
readiness  takes  account  of  the  downtime  after  the  system  becomes  nonoperative 
and  it  must  be  repaired.  Such  an  approach  takes  Into  account  cost  of  repair 
and  the  cost  of  downtime,  as  well  as  the  reliability  before  being  launched. 

This  approach  thus  Incorporates  the  notion  of  life  cycle  cost,  which  is  very 
Important  in  the  selection  of  weapon  systems,  since  the  total  resource 
requirements  must  be  considered  in  making  the  selection. 

3.  You  have  ignored  the  burn-in  or  infant  mortality  in  your  replacement 
decision.  The  need  is  not  only  to  be  in  the  wearout  portion  of  the  curve, 
but  also  to  be  far  enough  up  it  to  outweigh  the  risk  of  infant  mortality. 

4.  Also  the  choice  in  maintenance  is  sometimes  between  inspect  the  weapon 
system  or  replace  particular  components  on  the  basis  of  age  or  activity. 

You  may  want  to  inspect  a  component  before  you  replace  it,  especially  if 
there  is  uncertainty  as  to  the  rate  of  wearout.  There  is  a  well  developed 
theory  in  this  atea. 

5.  However,  there  are  still  complications  in  applying  a  parts  replacement 
policy  during  a  test  period.  This  is  because,  typically,  for  the  usual 
length  of  test  experience,  we  may  get  no  failures,  yet  we  know  that  the  parts 
will  fail.  This  problem  is  equivalent  to  that  of  observing  very  low  demands 
In  inventory  theory.  To  deal  with  the  problem  of  0  demand,  we  usually  have 
to  inject  some  prior  probability  distribution,  like  a  negative  exponential 

to  get  a  practical  parts  replacement  policy. 

6.  The  Barlow-Proschan  theory  picks  up  the  notion  of  monotonlc  failure 
distributions.  They  have  a  bock  on  Mathematical  Theory  of  Reliability, 

plus  many  papers,  including  empirical  analyses,  to  illustrate  their  technique. 

7.  It  would  also  be  worthwhile  to  explore  the  possibility  of  using  computer 
simulation  to  get  at  the  desired  confidence  intervals  by  sampling. 

8.  Subsequent  to  Che  conference,  two  references  were  located  that  it  is 
felt  should  be  helpful  to  the  author  in  his  work.  These  are:  Mathematioal 
Methods  of  Reliability  Theory,  by  B.  V.  Gnedenko,  Yu  K.  Belyayev,  and  A.  D. 
Solovyev,  published  by  the  Academic  Press,  1969;  and  'onfidcnae  Interval 
Estimation  of  the  Reliability  of  Multicomponent  Systems  Using  Component 
Test  Data,  ARDC  Technical  Report  No.  3,  by  Jerome  R.  Johnson,  published  by 
the  Aberdeen  Research  and  Development  Center  in  November  1969. 
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TEST  PROCEDURES  FOR  EVALUATION  OF  NUCLEAR  EFFECTS  ON  INITIATORS 


W,  B,  Thomas  and  R.  E.  Beets 
Solid  Propellant  Chemistry  Branch 
Army  Propulsion  Lab 
Redstone  Arsenal,  Alabama 

The  exoatmo spheric  explosion  of  a  nuclear  device  produces  radiation 
which  la  absorbed  by  stater lals.  The  amount  of  radiation  absorbed  by  any 
given  material  is  roughly  equivalent  to  the  fourth  power  of  its  atomic 
number.  The  absorption  phenomena  produces  a  high  temperature  gradient  in 
such  a  short  time  that  it  generates  a  shock  wave  in  the  material  causing 
spallation  and  fracturing  of  metal  parts,  and  activation  of  explosive 
components. 

The  following  VU-GRAPHS  shew  graphically  this  phenomena.  Figures  1 
and  2  show  by  relative  areas  the  amount  of  this  energy  absorbed  by  given 
materials  with  respect  to  iron.  The  division  being  made  to  those  materials 
used  by  conventional  initiators  and  those  materials  proposed  for  use  with 
the  Low-Z  SQUIB. 

Figure  3  shows  the  deposition  phenomena  and  accompanying  reaction  with 
respect  to  time;  the  deposition  time  occurring  in  fractions  of  microseconds. 
Figures  4  and  S  show  the  reaction  as  it  would  occur  in  an  initiator. 

Electric  initiators  have  been  exposed  to  simulated  nuclear  environments 
in  underground  testing.  Both  Plcatlnny  Arsenal  and  Redstone  Arsenal  have 
participated  in  these  tests.  Since  conventional  electric  initiators  are 
composed  of  noble  metal  electric  bridgewires,  and  in  some  instances  explo¬ 
sives  with  high  atomic  number,  such  as  lead  azide  or  lead  styphante,  it  is 
obvious  that  these  initiators  would  be  quite  susceptible  to  the  radiation  of 
the  bomb.  Tests  have  proven  this  to  be  the  case. 

With  a  relative  amount  of  energy  in  the  order  of  (1),  the  lead  compound 
initiators  explosives  detonate.  Between  the  relative  numbers  of  3  to  10, 
explosives  of  low  atomic  numbers  will  initiate  if  they  are  adjacent  to  the 
bridgewire  which  are  f  high  atomic  number  such  as  gold  or  platinum.  In 
the  relative  number  30  range,  these  wires  will  actually  be  vaporized  to 
the  extent  that  they  will  be  detrimental  even  if  no  explosive  were  in  con¬ 
tact  with  the  wires. 

The  design  approach  of  the  Propulsion  Leb  is  to  design  an  initiator 
which  Is  composed  of  low  atomic  numbered  elements.  Primarily  the  goal 
would  be  to  use  elements  of  atomic  number  (13  Aluminum)  and  below. 
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A  prototype  design  of  this  initiator  wee  assembled  and  tested  In  an 
underground  test,  and  shovn  to  be  able  to  survive  the  relative  30  level. 

This  design  was  considered  primeval  In  nature  and  proved  only  the  feasibility 
of  the  concept.  Development  effort  on  this  initiator  has  remained  dormant 
for  approximately  2  years  duo  to  the  Inability  to  schedule  underground  tests. 
However,  within  recent  weeks  scheduling  for  such  teats  have  been  acquired 
and  developmental  effort  has  bean  reactivated. 

This  effort  will  now  cotmsence  with  a  more  sophisticated  design  using  a 
metal  to  cerauic  header  with  low-Z  brldgewlras  or  spark  gap.  The  materials 
of  prlion'-’y  interest  ere  aluminum-alumna  header  and  berylllum-barylla  header 
with  beryllium  or  aluminum  bridgavire.  In  addition  to  the  electrical  Initiator 
approach,  effort,  la  also  being  expended  to  develop  an  initiator  which  is 
activated  by  laser  light.  The  concept  la  to  use  laser  energy  piped  through 
fiber-optics.  The  laser  energy  Is  transmitted  into  the  initiator  through  a 
quarci  window.  Hopefully,  the  element  ailicon  will  be  the  highest  Z  number. 

In  the  past,  in  order  to  obtain  data  as  to  the  survivability  of  these 
initiators  to  the  nuclear  stlmulll,  a  giver,  number  of  Initiators  were  pieced 
at  a  given  distance  from  the  source.  Then,  baaed  upon  the  number  of 
initiators  which  survived  at  a  particular  station,  conclusions  were  drawn 
aa  to  the  survivability  of  the  item.  For  example,  if  all  the  Items  survived 
ac  1  and  3  relative  numbers  but  one  or  more  of  the  Initiators  were  damaged 
or  were  activated  at  the  10  relative  number,  then  it  wee  concluded  that  at 
relative  level  3  the  initiators  were  sale;  somewhere  above  3  but  under  10  they 
were  not.  At  present,  we  have  no  methcds  to  determine  the  quantitative 
effect  of  the  nuclear  environment  on  Che  initiators.  Therefore,  we  are  basing 
our  »urvlvablllty  numbers  on  GO- NO-GO  data  only. 

The  shortcoming  of  the  testing  program  appear  to  be  four-fold.  1.  The 
underground  teat  dose  not  give  a  true  reproduction  of  an  wxoatmoapheric 
environment.  2,  Quantitative  effects  csnnot  be  measured.  3.  The  expense 
of  performing  the  test  is  enormous,  4,  The  long  term  effects  are  unknown. 

When  the  two  initiators  now  under  development  reach  the  stage  of  com¬ 
pletion  for  testing,  our  plans  are  to  do  testing  similar  to  that  which  has 
already  been  done.  That  is,  to  use  a  sample  sice  of  10  to  20  initiators  and 
station  them  at  3  to  10  different  stimuli.  Figure  6  19  a  schematic  of  a  test 
set-up  showing  the  radiation  as  It  impinges  on  the  various  stations.  The 
lowest  station  at  which  damage  or  activation  does  not  occur  will  be  assumed 
as  the  safety  level,  if  after  exposure  the  Initiator  performance  does  not 
change.  The  space  in  an  underground  test  will  probably  be  limited  to 
approximately  25  sp.  in.  per  stetlon,  for  3  to  10  stations.  This  space 
limitation  fixes  the  10  to  20  sample  size.  The  question  posed  here  is,  "Is 
our  rationale  Justified  In  testing  with  such  low  numbers  in  determlng  thresh¬ 
old  levels  or  survivability  tc-  a  given  nuclear  environment?"  We  also 
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ask  which  ace  the  most  advantageous  atatlona  (or  testing  If  we  estimate  prior 
to  testing,  the  level  of  survival,  l.a..  If  va  estimate  chat  the  initiator 
will  survive  the  30  level  would  it  be  better  to  teat  two  stations  below  30 
and  five  stations  above  or  vice  versa? 

If  development  time  permits,  prior  to  underground  testing,  than  simula¬ 
tion  techniques  with  sophisticated  machines  will  be  used  to  determine  the 
threshold  laval  and  survivability  point.  This  tasting  will  be  accomplished 
by  the  Bruceton  method  in  which  approximately  25  to  50  Initiators  will  ba 
tasted.  The  energy  level  will  be  varied  and  the  SOX  point  calculated.  The 
O.SX  and  the  99. 5X  reliability  will  be  calculated  at  a  95X  confidence  level 
using  the  Bruceton  type  data. 

We  conclude  by  asking  what  is  the  most  advantageous  test  method  and  the 
best  method  of  handling  test  data  for  determining  the  adverse  nuclear  effects 
on  Initiators. 
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COMMENTS  BY  MURRAY  A.  GEISLRR 

OB 

"Test  Procedures  Cor  Evaluation  of  Nuclaar  Effaces  on  Initiators" 

by 

W.  B.  Thomas  &  R.  E.  Batts 


1.  The  sample  sires  seem  very  small  for  the  complexity  of  phenomena  to 
vhich  you  are  extrapolating.  I  would  want  supplementary  data  and  evidence 
to  develop  the  final  test  plan. 

2  The  possibility  of  using  simulation,  as  described  in  next  to  las*'  para¬ 
graph  of  the  paper,  seems  intriguing.  If  this  simulation  haa  validity,  I 
would  want  to  use  it  to  plan  test.  If  simulation  data  do  heir  to  determine 
the  threshold  value  and  survivability  point,  I  would  use  simulation  pre¬ 
dictions  to  help  eet  up  the  test  plan.  Simulation  data  could  also  help  to 
estlmete  the  teat  confidence. 

3  Using  simulated  data,  I  would  concentrate  teats  around  where  the  30 
level  seams  Indicated,  with  somewhat  more  tusts  distributed  above  the  30  level, 
to  be  more  confidant  chat  you  have  reached  it.  Concentration  around  where 

you  think  the  30  lavel  is,  will  depend  on  how  confidant  you  ere  in  the  simulation 
date. 


4.  1  would  also  try  to  calibrate  simulation  from  the  test  data,  and  might 

try  to  design  the  experiment  for  obtaining  teat  data  to  accomplish  this, 
since  apparently  you  can  replicate  the  simulations  more  readily.  I  would 
use  this  approach  as  s  primary  source  of  data  and  analysis. 

3-  You  still  have  the  problem  of  extrapolating  from  the  test  to  the  real 
world.  You  have  go-n©»go  data  only,  and  only  a  small  aample.  How  are 
initiators  distributed  as  to  material?  Can  you  estimate  effect  from  one 
material  to  another  ualng  this  4th  power  rule?  Do  distance  effects  get 
realistically  translated  through  using  this  rule  serosa  materials?  I  guess 
this  part  of  the  theory  is  still  very  tenuous. 
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OPTIMAL  DESIGNS  WITH  A  TCHEBYCHEFFIAN  SPLINE  REGRESSION 


V.  N.  Murty 

The  Pennsylvania  State  University 
Middletown,  Pennsylvania 

This  talk  will  be  a  summarised  version  of  three  papers  by  the  author. 
[See  references  below  as  to  where  the  full  papers  can  be  found.) 

Studden  (1968)  showed  that  the  optimal  design  for  estimating  any 
specified  regression  coefficient  or  parameter  is  supported  by  one  of 
two  sets  of  points  for  Tchebychef fian  systems  with  certain  symmetry  proper¬ 
ties.  In  the  first  paper  we  consider  a  Tchebychef flan  Spline  Regression 
Function,  defined  on  an  Interval,  and  show  that  the  optimal  design  for 
estimating  any  specified  regression  coefficient  is  supported  on  the 
same  set  of  points. 

When  an  experimenter  is  interested  in  more  than  one  parameter  in 
the  regression  model,  and  tries  to  obtain  a  design  that  minimizes  the 
maximum  variance  of  the  individual  regression  coefficients  he  is  looking 
for  a  mlnlmax  design  with  respect  to  the  single  parameter,  a  concept 
introduced  by  Elfving  (1*159).  In  our  second  paper  we  explicitly  present 
the  mirlmax  s.p.  designs  for  the  ordinary  polynomial  regression,  when  the 
degree  is  <_  12.  A  general  solution  of  this  problem  is  still  open,  but 
the  results  obtained  do  indicate  the  direction  in  which  one  could  look 
for  a  possible  general  solution. 

In  the  third  paper  it  is  shown  that  the  optimal  design  for  estimating 
any  specified  parameter  in  a  polynomial  spline  regression  with  a  single 
multiple  knot  at  the  center  is  supported  by  one  of  two  sets  of  points. 


REFER ENCES 

1.  Murty,  V,  N.,  OPTIMAL  DESIGNS  WITH  A  TCHEBYCHEFFIAN  SPLINE  REGRESSION, 
Vol.  42,  No.  2,  April  1971  issue  of  Annals.  Ma t h .  Stat . 

2.  Murty,  V.  N.,  OPTIMAL  DESIGNS  WITH  A  POLYNOMIAL  SPLINE  REGRESSION  WITH 
A  SINGLE  MULTIPLE  KNOT  AT  THE  CENTRE,  Vol  42,  No.  3,  June  1970  issue  of 
Annals .  Math.  Stat . 

3.  Murty,  V.  N.,  MINIMAX  DESIGNS,  June  1971  issue  of  Jour.  Amer.  Stat.  Assn. 
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ORDINARY  AND  EMPIRICAL  3AYB3  APPROACH  TO  ESTIMATION  OP 
RELIABILITY  IN  THE  V.T.IEUI.L  LIRE 


TESTING  MODEL 


George  C.  Canavos 
NASA  Langley  Re Gear ch  Center 

and 

Chris  P.  i'sokos 
Virginia  Polytechnic  Institute 

SUMMARY 

An  ordinary  an  well  as  an  empirical  Bayesian  analysis  for  a  random 
scale  parameter  In  the  Veibull  distribution  is  developed  with  respect  tc 
the  usual  life  testing  procedures.  The  results  of  Bhnttachnryn  (19^7) 
are  generalized  to  include  the  flexibility  cf  the  shape  parameter  in  the 
Weibull  case.  Empirical  Bayes  estimators  are  developed  for  the  scale 
parameter  and  the  reliability  function  based  upon  prior  experiences.  A 
simulation  procedure  is  carried  out  and  a  comparison  is  made  between 
Bayes,  empirical  Bayes,  and  minimum  variance  unbiased  estimators. 

1.  ITSTECDUCTIOIl 

For  the  Weibull  failure  distribution  with  probability  density  function 


f(x|0,|)  *  |  x'"1  e  9  (0  <  ::  <  ■»,  0,  ■  >  0), 


(1.1) 


we  develop  ordinary  as  well  as  empirical  Bayes  estimators  cf  the  scale 


parameter  and  the  reliability  function 


-it* 


R(t|9,|)  «  e 


t  >  0 


(1-2) 


with  respect  to  the  use:  I  life-testing  procedures.  The  shape-  parameter 
|  is  assumed  to  be  known. 

This  paper  has  been  reproduced  photographically  from  the  author’s  manuscript. 


Preceding  page  blank 


Best  Available  Copy 


Since  in  many  life-testing  situations  it  is  not  unlikely  to  note  the 
unpredictable  fluctuation  of  the  scale  parameter  in  a  failure  model,  it 
is  Justifiable  to  consider  such  a  parameter  as  a  random  variable  and,  thus, 
appeal  to  a  Bayesian  analysis.  In  specific,  let  9  denote  the  random 
variable  associated  with  the  scale  parameter  and  0  its  realisation. 
Obviously,  a  Bayesian  analysis  depends  on  the  utilization  of  prior  infor¬ 
mation  which,  in  this  case,  we  assume  to  exist  either  in  the  form  of  a 
prior  distribution  of  0  or  a  sequence  of  sufficient  statistics  from  past 
experiments.  For  the  ordinary  Bayes  approach,  we  appeal  to  a  well-known 
transformation  to  generalize  the  results  of  Bhattacharya  (1967)  for  the 
one-parameter  exponential  model  so  as  to  include  the  flexibility  provided 
by  the  shape  parameter  |  in  the  Veibull  distribution.  In  fact,  the 
Weibull  failure  model  has  an  increasing  (|  >  1)  or  decreasing  (|  <  i) 
failure  rate  and,  thus,  is  likely  to  describe  the  life  span  of  items 
with  variable  failure  rates. 

The  empirical  Bayes  estimation  technique  was  largely  motivated  by 
Robbins  (1955)  who  assumed  the  existence  of  a  prior  distribution  for  an 
unknown  parameter  but  not  the  knowledge  of  its  form.  Instead,  he  substitutes 
past  information  which  he  assumes  to  exist  as  a  result  of  the  repetitive 
nature  in  the  problem  of  estimation.  Thus,  in  the  absence  of  knowledge 
concerning  the  form  of  the  prior  distribution,  we  appeal  to  an  empirical 
Bayes  approach  to  estimate  the  scale  parameter.  3y  using  this  estimate, 
an  estimate  cf  the  reliability  function  is  made  possible. 

2.  ORDINARY  BAYES  ESTIMATION 

iS*'  usual  procedure  in  life-testing  is  to  place  n  items  on  a  life- 
test  and  to  terminate  the  experiment  after  a  predetermined  r  5  n  number 
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of  failures:  have  been  observed.  Inferences  are  usually  based  on  a 


r 

sufficient  statistic  vhich  for  the  Weibull  model  is  given  by 

i-1 

where  (x^,X£,  •  •  • ,  xr)  =  x  denote  the  r  lifetimes  of  the  censored 
sample.  Clearly,  the  complete  sample  is  realized  for  r  =  n.  with 
respect  to  this  procedure ,  Bhattacharya  (1967)  considered  a  random  scale 
parameter  9  in  the  exponential  failure  distribution  with  density 


1 

-£y 


f(y|e)  =  i  e  0  (0  <  y  <  »,  0  >  0) 

& 


(2.1) 


and  for  the  usual  assumptions  derived  Bayesian  estimates  of  0, 

-  —  t 

R(t)  -  c  9  ,  and  their  variances  for  a  uniform,  an  exponential,  and  an 
inverted  gamma  prior  distributions.  Here,  we  extend  his  results  to 
include  the  Weibull  failure  model  and  the  flexibility  provided  by  its 
shape  parameter. 

It  is  well  known— see  Tate  (1959)— that  if  the  random  variable  Y 
has  the  exponential  density  given  by  (2.1)  then  the  random  variable 
X  =  follows  a  Weibull  distribution  with  density  given  by  (l.l). 

Hence,  for  the  Weibull  case,  the  Bayesian  estimates  and  their  variances 
which  are  given  in  tabic  2.1  follow  immediately  from  this  transformation 
and  the  results  of  the  exponential  model.  For  detail  of  proof,  see 
Bhattacharya  (lyJ?)-  We  remark  that  in  table  2.1, 

■  z 


and 


?(n,z)  =  j  e^t11"1  dt 
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represent  relations  involving  the  incomplete  gamma  function  while 

K  (an)  is  the  modified  Bessel  function  of  the  third  kind  of  order  v. 
v 

3.  EMPIRICAL  BAYES  KCTIMATIOH 

We  implement  here  an  empirical  Bayes  approach  for  the  estimation 
of  the  scale  parameter  and  the  reliability  function  in  the  Vcibull  distri¬ 
bution.  The  empirical  Bayes  method  that  ve  consider  was  introduced 
by  Bobbins  (1955),  and  its  theory  and  techniques,  were  developed  by 
Bobbins  (19^'*),  and  Rutherford  and  Krutchkoff  (l?69 ) ,  among  others.  The 
difference  between  empirical  Bayes  and  ordinary  Bayes  is  that  the  former 
does  not  make  explicit  the  form  of  the  prior  distribution  in  order  to 
make  possible  a  Bayes  solution.  Instead,  the  empirical  Bayes  method 
depends  or.  the  existence  of  prior  information  in  the  form  of  past 
estimates  of  either  the  parameter  in  question  or  some  c'osc  variation 
of  it. 

It  is  well  known  that  for  a  squared  error  loss,  the  Bayes  risk  is 
achieved  if  the  decision  function  ic  the  posterior  expectation, 

E^OIx},  where 

*  er> 

E{e|x)  =  /  9h(C|v)dS(9)  (5.1) 


past  -riences  to  provide  a  consistent  estimate”  of  the  posterior 
expectation  (3.1)  whose  risk  in  general  converger,  to  that  of  ordinary 
Bayes  as  the  nutter  cf  past  experiences  used  tends  toward  . 
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In  the  current  situation,  the  existence  of  prior  information  1b 

based  on  ihe  assumption  that  lifc-teBts  have  been  conducted  periodically 

on  identical  items  and,  thus,  past  failure  information  has  accrued. 

In  specific,  ve  assume  the  existence  of  k  >  1  past  sufficient  statistics. 

bet  {xp>Tj_)>  (£2^2) *  . represent  the  sequence  of  previous 

information  where  the  vector  Xj  =  (xpj,  x^j,  •••,  xnj)  denotes  a  sample 

n 


of  n  >  1  random  lifetimes  from  (l.l),  and  Tj 


represents 


J-l 

the  corresponding  sufficient  statistic.  We  use  the  fact  that  for  known 
shape  parameter  £,  T  is  a  sufficient  statistic  of  fixed  dimensionality— 
Raiffa  and  Schlaifer  tl9el)--and  thus  find  an  equivalent  form  of  the 
posterior  density  (3.2)  conditional  on  the  sufficient  statistic  T 
rather  than  upon  x.  We  determine  first  the  probability  density 
function  of  T  conditional  on  0  which  is  given  by 


q(T|e)  -  — L__  Tn“1e“T/°  (0  <  T  <  »,  0  >  0).  (3.3) 

r(n)0 

This  follows  from  the  fact  that  if  X  is  Weibull  then  X5  fellows  an 
exponential  distribution;  therefore,  the  density  of  T  is  the 
convolution  of  n  exponentials.  See  Tate  (1959). 

Notice  that  the  likelihood  of  a  sample  of  lifetimes  from  (l.l) 
can  be  written  as 


l(xle) 


_n-3  -T/o  5  r(n).H  Xj 
T  e  '  x-x 


«-l 


T(n)9n 


n-1 


which  reduces  to 


Z(x|0)  ^  q(Ti O)r(x). 
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If  the  prior  density  of  6  is  c,(0),  the  Joint  probability  density  of 
x  and  C  \  Is 

:'(x,0)  -  q(?ie)g(o)r(x)  -  q'(T,e)r(x).  (J.U) 

Integratliy;  (J.h)  over  S,  we  obtain 

**(x)  *  <l*(T)r(x) 


where 


q*(T)  »  /  q(Tio)  dc(o). 

”  O 

As  a  result,  the  posterior  uenslty  h(8|x)  can  to  written  as 

l'(x,e)  q(TI0)c(0) 

MOIx)  =  — — —  =  - — -  -  h(OiT). 

1  (x)  <1  (T) 

Therefore, 


E(s.x}  --  E(eiT)  *•  /  0h(9lT)  de. 

^  o 


(3-5) 


The  cor.dltloi.nl  density  of  where  T'^  denotes  the  current 
sufficient  statistic  can  be  vritter. 


«r.(Ve> 


(3.6) 


where  the  subscript  n  implies  that  is  a  function  of  n  random 

life  tir.es.  Multiplying  equation  (5-6)  by  9  and  after  oorr.e  r.nnlpulntlon, 
thv.  *_y..  1  r  -educed  to 
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-  rrr  •  (3-7) 


where 


«WV9>  *• 

ie  the  evadltlor.nl  density  of 
lifetimes  of  the  current  sample, 
equation  (3-*))  we  obtain 


as  n  function  of  the  first  (n  - 
Substituting  equation  Lu 


D 


<®ITk} 


JQ  Vi(Tk>9) 


or 


Tk  fc.l(Tk)\ 
(n  "  1}  $(Tk>  J 


(3.8) 


Obviously,  the  marginal  densities  q^  ^T^)  8X1,1  depend  on  the 

form  of  the  prior  distribution  0(6).  However,  by  substituting  con¬ 


sistent  estimates  q*_^  s(Tk)  and 

an  empirical  Bc.yes  estimator  of  e 
of  such  an  estimator  is  reduced  if 


q*  k(Tk),  respectively,  we  obtain 


.  Based  on  empirical  evidence,  the  risk 
the  quantity  T-  jq  r— l^k(7K)  |  is 

lq*,„k(Tk)  t 


divided  by  r.  rather 
Bayes  estimator  of  the 


har.  by  (r.-l).  Hence, 
current  realisation 


the  recommended  empirical 
0k  becomes 

(3.9) 


Substituting  (3*v)  in  (1.2),  wc  obtain  an  estimator  of  the  reliability 
function  based  on  ?k  and  expressed  by 


RMV5) 


t  >  0. 


(3 1' ; 
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For  the  computation  of  q^_-j  ^(T^)  and  q*  wc  appeal  to  a 

result  of^Parzcn  ( iyC2)  who  has  provided  a  class  of  consistent  estimators 
of  probability  density  functions.  In  specific,  we  recommend  an  estimator 

*  t. 


wnere 


*  -lo* 

J 


for  )/'!„( Tk)  of  the  form 

)  ’  e”‘  ^ 

Vi;k(Tk)  jri 
*ln,k(Tk)  J'  ^bjl 
j=l 


Tn,k  "  T(n-l),j 


* 


^n.k  '  ir 


(5-11) 


j  -  1 }  y  «  •  •  }  " 


and 


5  =  k 


-1/5 

\J 


rw 

\  (T  ,  -  T  f. 

U  n,j  n 


J.=T 


4 .  A  SttlULATIOu  PRCCEPUiff; 

From  the  results  of  the  previous  section,  the  validity  of  an  empirical 
Bayes  estimator  rests  on  the  notion  that  its  risk  approaches  the  risk  of 
ordinary  Bayes  as  the  amount  of  past,  information  becomes  subst:mtial. 
However,  if  the  risk  of  empirical  Bayes  does  not  converge  to  tint  of 
ordinary  Bayes  within  a  relatively  small  number  of  past  experiments,  the 
practical  application  of  the  former  in  life  testing  seems  dcul.tf  1.  pair 
is  primarily  because  of  the  expected  limitation  , a  the  sir.e  of  past 
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information.  Moreover,  if  the  size  of  pant  information  in  inured 
substantial,  say  15  or  more,  the  reasonable  approach  in  such  a 
situation  should  be  based  on  an  attempt  to  identify  the  prior  distri¬ 
bution  form  and  use  ordinary  Bayes  procedures  rathe:-  than  empirical 
Bayes  methods.  Hence,  ve  find  it  necessary  to  appeal  to  a  Monte 
Carlo  simulation  of  random  lifetimes  from  the  tie 1 bull  distribution 
ft  that  a  comparison  of  small -sample  behavior  of  ordinary  Bayes  and 
empirical  Bayes  be  made  possible.  At  the  same  time,  ve  investigate  the 
degree  of  improvement  of  the  empirical  Bayes  estimators  (3.?)  and  (3.10) 
as  functions  of  the  number  k  of  past  estimates  over  the  corresponding 
minimum  variance  unbiased  (MTU)  estimators  developed  by  Tate  (1959). 

The  simulation  procedure  is  as  follcvs:  i-  complete  sample  of  n 
lifetimes  is  generated  by  first  simulating  a  realization  0  according 
to  a  prior  distribution.  This  task  is.  repeated  15  times.  At  each 
time,  the  squared  deviations  from  9  and  the  reliability  function  of 
ordinary  Bayes  and  MVU  estimates  are  computed  and  stored  vith  the 
reliability  estimates  calculated  for  relatively  small  increments  of  t. 
For  k  =  2,  5,  10,  and  15,  the  corresponding  empirical  Bayes  estimates 
and  their  squared  deviations  are  determined  3S  functions  of  the  k 
previous  estimates.  Finally,  the  entire  process  is  repeated  500  times 
and  average  squared- errors  are  computed.  The  criterion,  for  comparison 
is  determined  by  considering  mean-squared  error  (MSK)  ratios  of 
empirical  Bayes  to  ordinary  Bayes  and  to  MVU  estimators. 

The  results  cf  the  Monte  Carol  si"  ..at  ion  are  systematically 
given  in  t’r.-.  tables  below.  The  situation  under  which  the  contents 
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of  each  table  were  realized  is  made  explicit  and  need  not  be  repeated 
here.  We  remark  that  an  index  to  the  improvement  of  the  empirical 
Bayes  estimator  over  the  corresponding  MVU  estimator  is  provided  by 
the  quantity 

E{Var(e|e)} 

A  =  - , 

Var(9) 

where  Var(§|o)  is  the  conditional  variance  of  the  sufficient 

*  m 

estimator,  9  =  and  Var(9)  is  the  prior  variance. 

The  following  general  conclusions  drawn  from  the  results  given 
below  are  apparent  for  a  random  sca?e  parameter  in  the  Weibull 
distribution: 

(1)  As  expected,  the  ordinary  Bayes  estimators  have  the  smallest 
MSE  irrespective  of  prior  distribution  form,  prior  parameter  values, 
or  shape  parameter  value. 

(2)  The  MSE  of  the  empirical  Bayes  estimators  gets  closer  to 
that  of  ordinary  Bayes  as  the  number  of  past  estimates  increases. 
However,  it  is  doubtful  whether  further  squured-error  improvement  can 
be  made  with  more  than  15  past  estimates. 

(3)  Generally,  for  k  >  5  past  estimates,  the  empirical  Bayes 
estimators  have  smaller  MSB’s  then  corresponding  MVU  estimators. 
Moreover,  the  rate  of  improvement  is  not  a  function  of  prior  distri¬ 
bution  form  but  is  directly  proportional  to  the  index  A. 
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EQUI -RADIAL  DESIGNS  FOR  WEIGHTED  REGRESSION 


ANALYSIS;  FITTING  A  SECOND-DEGREE  MODEL. 


John  A.  Cornell 
University  of  Florids 


When  constructing  design  configurations  for  the  purpose  of 
exploring  a  responte  surface,  one  often  assumes  that  the  obser¬ 
vations  possess  homogeneous  or  constant  variation  within  the 
experimental  region.  Designs  introduced  by  Box  and  Draper 
[1959,  1962]  and  others  have  received  considerable  attention. 

We  relax  the  assumption  of  homogeneous  variation  among 
the  observations  by  partitioning  the  region  of  interest  into 
two  variance  tones  where  if  an  observation  is  collected  from 

2  2  2 

tone  i  (i«I,II)  it  has  variance  and  0  «*  ko^  ,  k  *  .25,  .5, 

1.0,  2.0,  4.0.  The  designs  discussed  are  second  and  third- 
order  equi-radial  rotatable  designs.  A  weighted  least-squares 
analysis  is  used  enabling  us  to  compare  configurations  which 
are  optimum  when  k  /  1.0  to  optimum  configurations  when  it  is 
assumed  the  variance  of  the  observations  is  constant  or  k  *  1.0. 
This  article  has  been  reproduced  photographically  from  the  author's  manuscript. 
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1.  introduction 


when  exploring  a  response  surface  using  the  method  of  Box  and 
Wilson  C 5 j #  the  following  assumptions  are  usually  madet 

(i)  the  true  surface  can  be  expressed  as  a  function  of  k 

controllable  factors  x^»  X^ . X^  and  at  the  uth  sample 

point  (u«l,2, . . . ,N) ,  the  true  surface  is  written  as 


where 
the  u 


th 


1U 


*  *  (X,  ,x„  , . . . ,X^  ) 
j  lu  2u  ku 

.s  the  value  (setting)  of  the 


sample  point: 


.  th 

l 


factor  at 


(1.1) 


(ii) 


(iii) 


within  a  given  region  of  interest,  the  response  surface 
may  be  represented  by  its  Taylor's  expansion  to  term3 
of  order  d. 


S^X.+S  v  +. 
0  0  11 


'+3UXl+---+fi12XlX2+- 


+'lllXl+- 


(1.2) 


where  in  the  subscript  of  3.  the  number  of  times  each 
factor-number  appears  is  the  appropriate  power  of  that 
factor  (and  XQ=1) ; 


the  observed  response  y  varies  about  a  mean  of  h  with 

2  U 

a  common  variance  c  for  all  values  of  u,  these  N  errors 
being  uncorrelated,  i.e.. 


y  *  n  +  e 
7U  u  u 


(1.3) 


where 

E(«u)  =  0,  E(e^)  =  o2  ,  E(eyeut)  =  0  uj*u 1  ;  u,u‘=l,2 . N  (1.4) 

(iv)  the  estimate  of  the  value  of  the  response  at  some  point 

x‘  **  (X  ,X  ,...,X  }  in  the  region  of  interest  can  be 
1  2  K 

expressed  in  model  (vector)  form  as, 

y(x)  =  (1.5) 

where  x‘  =  ( 1 , X, , . . . , X^ , . . . ) ,  h*  =  (b  ,b  , . . . ,b  , . . . ) 

*"1  i  1  **1  0  1  11 

and  the  b^'s  are  estimates  of  the  B^'s  in  (1.2). 
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In  an  attempt  to  approximate  the  relationship  (1.2)  with  a 
polynomial  model,  one  performs  experiments  at  N  predetermined 
combinations  of  the  levels  of  the  k  controllable  factors.  These 
combinations  are  referred  to  as  the  experimental  design.  Once 
the  N  observations  are  collected,  the  b^'s  in  (1.5)  may  be  obtained 
by  the  method  of  least  squares  using  the  formula. 

Si  •  (Whsi*  •  <16> 

where  is  the  matrix  of  values  taken  by  the  terms  in  xj  over  the  N 
experimental  combinations  of  the  k  factors  and  ^  is  the  column  vector 
of  the  N  observed  response  values.  (We  have  assumed  to  be  ncn- 

singular . ) 

For  response  surface  exploration,  a  great  deal  of  information 
is  available  about  a  certain  class  of  designs  called  rotatable 
designs.  Such  information  may  be  found  in  the  following  papers, 
[1,2,6,7,8,9,10].  For  the  present  work,  however,  the  papers 
[8,10,ll]  provide  most  of  the  necessary  background  in  that  we 
shall  be  concerned  primarily  with  second-order  and  third-order 
rotatable  designs  in  two  controllable  factors. 

2 .  An  application 

In  an  industrial  combustion  stack,  the  variability  of  the 
particulate  matter  is  not  uniform  across  the  stack  as  it  approaches 
the  opening  at  the  top  of  the  stack.  This  lack  of  uniformity  of  the 
variance  is  owing  to  the  swirling  motion  of  the  particles.  The 
swirling  action  of  the  particles  causes  not  only  a  greater  concen¬ 
tration  of  particles  but  also  greater  particle  variation  near  the 
sides  of  the  stack  than  at  the  center.  An  illustration  of  the 

variation  profile  for  a  particular  case  is  shown  in  Figure  1. 

2 

(On  a  scale  of  <?  =1) 
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Figure  1.  Variance  profile  across  a  stack. 

The  particle  concentration  distribution  across  the  stack  is 
of  interest  to  us  for  sampling  reasons.  Hence,  if  we  wish  to  use 
response  surface  methods  in  an  attempt  to  describe  the  distribution, 
we  must  be  careful  about  making  the  assumption  of  homogeneous 
variation  (1.4).  We  now  consider  an  alternative  to  assumption  (iii). 

3.  Development  of  present  work 

We  shall  relax  the  assumption  of  homogeneous  variation  among 
the  N  observed  responses  £  and  instead  consider  the  following, 

E(c)  -  £  ,  (3.1) 

Var-cov(c)  »  E(««/ )  *  wo2 

where 

W  -  diagonal  (“  . . £  )  »  (3.2) 

W1  W2  WN 

is  not  necessarily  equal  to  w^,,  u/u’  . 

(When  wy“l  for  all  u»l,2,...,N,  (3.1)*(1.4)  and  (3. 3)*(1.6) ) .  The 
form  of  the  variance-covariance  matrix  (3.2)  will  amend  the 
estimation  equation  (1.6)  to  read. 
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(3.3) 


Si  '  'ttS'V'1*!*'1*  • 

Furthermore,  the  Variance-covariance  atructure  of  the  estimates 
will  take  the  form, 

Var-cov  (bx)  -  (3${vf1X1)“1°2  (3.4) 

The  formula  (3.4)  arises  in  the  method  of  weighted  least  squares 
(see  Cl2 ] ,  p.  77-81) . 

We  shall  not  be  concerned  directly  with  the  magnitudes  of  the 
individual  w^'s  in  (3.2),  but  rather  with  hjw  the  difference  in  the 
magnitudes  of  different  sets  of  the  w^  influences  the  construction 
of  the  design.  For  example,  in  Figure  2  a  cross-section  of  the 
combustion  stack  of  Figure  1  is  shown. 


i 


Figure  2.  Cross-section  of  the  stack. 

Since  we  know  that  the  observations  across  the  stack  do  not  have 

constant  variability,  we  attempt  to  simplify  the  problem  of  design 

construction  by  first  considering  the  cross-sectional  area  as  being 

comprised  of  two  variance  zones,  denoted  by  ZONE  I  and  ZONE  II.  If 

2 

an  observation  is  ken  from  ZONE  I,  it  possesses  a  variance  o  ; 

1  2 

whereas  an  observation  taken  from  ZONE  II  possesses  a  variance  of  0  . 

(Consideration  of  more  than  two  zones  is  discussed  later  in  Section  5.) . 

The  question  we  wish  to  answer  is  this: 

2  2 

If  o  «■  k<?  where  k  is  some  known  value  in  the  interval 
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(.25,  4.0)  how  should  the  experimental  design  configuration  be 
oriented  so  as  to  minimise  the  quantity, 

J  *  \  E{?(g)-n($)  ]2d£  (3.5) 

a  *r 

where 

O'1  ■  )  RJ*  .  (3-6) 

$  (g)  and  i^(g)  are  defined  in  (1.5)  and  (1.2)  respectively  and  R  is  the 
region  of  interest  (stack  cross-section  in  our  example) .  The  vector 
g  *  (X1#X2)'  contains  the  values  of  the  position  coordinates  across  the 
stack.  The  quantity  J  in  (3.5)  is  the  mean  square  error  of  $(£) 
averaged  over  the  region  R. 

By  configuration  orientation  is  meant?  for  a  given  value  of  k 
2  2 

where  o  =  k^,  and  using  two  or  more  sets  of  equi-radial  points, 
should  the  design  points  be  placed  at  the  extremes  of  the  two  zones? 
Should  the  sets  of  points  contain  large  n^  where  n^  is  the  number  of 
points  comprising  set  i?  We  now  discuss  the  minimization  of  J  using 
the  method  of  minimum  bias  estimation  (see  rn]). 

Development  and  Minimization  of  J 
It  is  easy  to  show  that  the  quantity  J  in  (3.5)  is  the  sum  of 
the  average  squared  bias  (B)  of  the  model  and  the  average  variance  (V) 
of  the  model,  that  is, 

J  *  B  +  V  (3.7) 

where  - 

B  *  -2  J  (Ery(x) ]  -  t(x) 12dx  (3.8) 

<3  R 

and  ^ 

V  -  -  J  Var  y (x)dx  .  (3.9) 

OR 
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At  expected,  we  can  concentrate  on  the  minimization  of  J  in  the  region 
of  interest  through  the  choice  of  the  experimental  design.  The 
orientation  and  size  (number  of  points)  of  the  design  will  of  course 
depend  on  the  relative  magnitudes  of  the  B  and  V  contributions.  In 
fact.  Box  and  Draper  [7,8]  show  that  to  minimize  V  alone,  one 
spreads  the  design  points  to  the  boundary  of  the  region  of  interest. 

To  minimize  B  alone,  however,  one  decreases  the  distance  from  the 
center  of  the  design  to  the  non-central  points.  Consequently,  when 
there  is  the  possibility  of  a  contribution  from  both  V  and  B,  one 
compromises  in  the  size  of  spread  of  the  points. 

A  different  approach  to  minimizing  J  is  discussed  by  Karson. 

Manson  and  Hader  [ill.  This  approach  involves  estimating  the  parameters 
in  the  fitted  model  to  achieve  minimum  B  in  (3.8).  Subject  to  achieving 
minimum  B.  one  then  concentrates  on  minimizing  V  in  (3.D)  through  the 
orientation  of  the  design.  We  now  discuss  this  approach  called 
minimum  bias  estimation. 

Let  the  fitted  equation  (second-order  model)  be  written  as 


y(*) 


x'b, 

~1~1 


where 


(l.X1.X2.XrX2.X1X2)- 


b’  -  (b0'bl,b2'bll'b22,b12) ' 


(3.10) 

(3.11) 


Assume  the  true  response  surface  is  represented  by 

r.  ■  v  1  5  +  x  '  5 

»1-1  -2-2 


where 


.2  3  ..2 


-2  “  (X1.X1X2,X2.X1X2)-  ;  =  ( " 1 1 T = 122 ' = 222 ' 5 1 1 2 ! 


(3.12) 


(3.13) 


Letting 

E[y(2) 1  =  x'£a  ,  say,  (3.14) 

it  is  easy  to  show  (see  C 1 1 1 ,  p.  464,  8];p.  349-350),  that,  to 
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minimise  the  value  of  B  in  (3.8),  that  la,  to  obtain 


whara 


wa  have  for  v, , 


Min  B  -  \&2^22  “  **12-11  »12*®J 

U 

jfij  -  n  5  SiSj  *«  j 


isj-1,2 


Zl  "  -1  +  -  1  ««12>2  * 


(3.15) 


The  matrices  u, .  in  (3.15)  are  called  region  moment  matrices.  (See 
-i  j 


[7,8]).  Rewriting  as 


v,  -  as  -  rT:u,r1u._]["|i"| 
-1  —  -.-11  -12  LB_J 

-2 


(34'! 


we  see  that  a  necessary  and  sufficient  condition  for  the  minimization  \Jv_ 
of  B  is  simply, 

E(^)  -  AS  .  (3.18) 

Since  E(y)  -  XS  whore  X  *  (X, Ix«),  and  if  we  express  b  as  a  linear 
combination  of  the  observations. 


,  \\ 


V  v  -  fl 


5i  3  Vl 

then  from  (3.18)  the  matrix  T*  must  satisfy, 

T’X  «  A  , 

00  (V  +0 

Thus  Min  B  can  be  obtained  as  long  as  AS  is  estimable. 

2  mm 

Since  Var-cov(y)  «  Jjc*’,  it  iB  worth  mentioning  that  designs 
satisfying  the  condition, 

(X.'W^x.  )“1X.,W_1X,  -  u  "lu,, 

-1—  -1  -1—  -2  -11  -12 

and  further  if  we  use  the  least  squares  estimator  (sec  (3.4)), 


(3.19) 


(3.20) 


(3.21) 


t'  -  (xnrVry-w"1 

m  **1  0*\*0 


(3.22) 
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to  that  J’g  *  the  weighted  least  squares  estimator  (3.4)  does 
achieve  Minimum  B. 

Subject  to  achieving  Min  B  using  the  estimator  (3.19)  where 

T'  -  MX'W-^^X'W-1  (3.23) 

we  now  attempt  to  minimise  V  in  (3.9)  through  the  orientation  of  the 
design.  The  quantity  is  minimized  when  $>(x)  is  the  .'east  squares 
estimate  of  its  expectation  at  all  points  in  R.  Si.v*  2f5(jc)  l^^-x^hg. 
then  V  is  minimised  when 


«i  -l  -I 
x.‘A(X'W  X)  X‘W  y 

mm  e  w  w  ^ 


(3.24) 


that  is,  when  T*  is  as  shown  in  (3.23).  Furthermore,  the  variance  of 
the  fitted  equation  is, 

Var  ?(x)  *=  x.'AfX'W ^Xj^A'X.ff2  (3.25) 

and  V  in  (3.9)  can  be  written  as 

V  -  trace Ta(X,w‘1X)~1A’u..1  ]  .  (3.26) 

We  now  discuss  the  design  configurations  to  be  used  for  the  purpose  of 
minimizing  J  in  (3.5).  The  designs  are  a  special  class  of  third-order 
rotatble  designs  in  two  variables  (see[10]). 


4 .  Designs 

As  we  mentioned  previously,  we  shall  consider  fitting  a  model  of 
the  second  degree  and  assume  the  true  polynominal  .  lationship  n  is  of 
the  third  degree.  The  design  configurations  used  will  be  rotatable 
designs  consisting  of  two  sets  of  equi-radial  points  lying  on  concentric 
circles.  The  inner  set  will  consist  of  n^a7  equi-radial  points  located 
at  a  distance  from  the  center  (X^O,  X^O)  of  the  design  space 

whereas  the  outer  set  of  points,  n^i7  in  number,  will  be  located  at  a 
distance  from  the  center  of  the  design  apace.  We  shall  also 


require  p  *  boundary  of  ZONE  I  and  p^*  boundary  of  ZONE  xx.  For 
simplicity  of  development,  ws  set  (^P^.S  and  ,5<Pj*1.0. 

Tha  usa  of  this  particular  class  of  designs  enables  us  to  specify 
the  form  of  the  syimatrical  matrix  of  Equations  (3.23)  and 

(3.26)  to  be  as  followst 


X’w“1X  - 


(Same) 


where  the  quantities  a.  b,  d  and  f  are. 


0 

b 


0 

0 

b 


b 

0 

0 

3d 


b 

0 

0 

d 

3d 


0 

0 

0 

0 

0 

d 


0 
3d 
0 
0 
0 
0 
5  f 


Ew 


b  -  Ex  w 


*  , 

iu  u 


d  *  Ex 


0 

d 

0 

0 

0 

0 

f 

f 


0 

0 

3d 

0 

0 

0 

0 

0 

5f 


2  „2 
,  X,  w 
lu  2u  u 


(4.1) 


(4.2) 


EX?  X?  w 
lu  2u  u 


EX*  X?  w 
lu  2u  u 


(i-1,2) 


and  the  summations  are  over  u»l,2 . N.  Furthermore,  using  n^  and 

in  place  of  X.  (i«l,2)  in  (4.2),  we  have  for  a,  b,  d  and  fj 

1  1  2  2 
a  -  ♦  n2w2  .  b  -  -  J^n.o.w. 


(4.3) 


d  ■  E  n .  c?w .  and  f 
8  i-i  111 


J7  E  n.P6w. 
16  i-1  1  1  1 


since  tne  (i-1,2)  ace  constant  for  all  points  located  at  a 

distance  o.  from  the  center  of  the  design. 

1  _! 

Pausing  briefly,  the  form  of  the  matrix  X'w  X  is  as  specified 

in  (4.1)  as  long  as  we  use  at  least  two  equi-radial  sets  of  points 

where  the  radii  of  two  or  more  sets  are  greater  than  zero  and  n^??. 

Of  course,  to  obtain  the  estimates  using  the  formula  (3.3)  we  would 

need  only  one  equi-radial  set  of  points,  the  radius  of  which  is 

greater  than  zero,  consisting  of  n*5  points  and  n^?!  additional 
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center  point  replicates.  With  this  latter  design  configuration 
however,  the  matrix  in  (4.1)  is  singular.  To  obtain  a  value 

of  V  in  (3.26),  the  use  of  a  generalised  universe  matrix,  the  form 
of  which  is  not  unique,  would  be  necessary.  This  would  complicate 
the  design  problem  considerably. 

A  further  note  to  consider  is  that  in  addition  to  the 
points  mentioned  above,  wo  could  run  nQ  center  point  replicates. 
These  additional  points  would  help  us  to  reduce  V  in  (3.26). 
However,  since  we  are  talking  already  about  designs  consisting  of 
et  least  14  points,  we  dismiss  the  notion  of  additional  nQ  center 
points . 

The  symmetric  matrix  u  (see  (3.15))  takes  the  form, 


4*11 


l  o  o 

c#2/4  0 

P*2/4 

(Same) 


c*2/4  c*2/4  o 

0  0  0 

0  0  0 

30*4/24  c*4/24  0 

3P*4/24  0 

5*4/*4 


(4.4) 


where  0*  is  the  radius  of  the  region  of  interest  (from  the  center  to 
the  boundary  of  ZONE  II).  Since  we  previously  set  0*  ■  1.0  and  from 
the  description  of  the  matrix  A  in  (3.17),  the  average  variance  V  in 
(3.26)  can  now  be  written  as,  (using  (4.1),  (4.3)  and  (4.4), 

1  a+24d-6b  I8f+b-12d  90f+7b-S4d 

V  “  TTd  * - 2  + - T  + - ~  •  <4- 

12 (2ad-b  )  72(bf-2d  )  72 (5bf-14d‘) 

Minimizing  V  in  (4.5)  then  is  analogous  to  minimizing  J  in  (3.5). 

Values  of  V  in  (4.5)  are  presented  in  Tables  1-5  corresponding 
to  the  following  values  of  n  ,  3,  and  k: 

ii 
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7,8,10 


n2  =  7,8,10 


-  .1,  .25,  .50  P2  -  .80,  .95,  1.0 

k  ■  0.25,  0.5,  1.0,  2.0,  4.0  . 

Only  included  in  Tables  1-5  however,  is  the  best  P2  value  for  each 
Pj  value  defined  above.  For  example,  when  k  *  0.5,  the  values  of 
V  When  P1  ■  0.1,  P2  ■  0.8  are  less  than  the  values  of  v  when  P^^  "0.1 

and  ^  »  0.95  or  P^  ■  0.1  and  P  =  1.0  for  all  corresponding  values 

of  and  n2>  Thus-  x  represents  V  at  »  0.1,  P  »  0.8  in  Table  2. 

In  Table  3,  the  value  of  k  is  k  *  1.0.  Table  3  provides  us  with 

an  insight  to  the  behavior  of  V  for  the  above  n^,  n^,  P^  and  P, 

values  when  it  is  assumed  the  variance  of  the  observations  is 
constant  in  the  region  of  interest. 

Table  1.  Values  of  V  when  k  s  0.25. 


JOB 

7 

8 

10 

n2  * 

7 

8 

10 

7 

8 

10 

7 

8 

10 

cm 

.71 

.61 

.75 

.68 

.58 

.72 

.65 

.55 

IB 

.69 

.59 

.74 

.67 

.57 

.71 

.64 

.54 

Efl 

.70 

.66 

.67 

.64 

.60 

.59 

.56 

.51 

x:  P1*0.1,  P2**0 . 8  P1=0.25,  P2=0.8  o:  P^O.5.  P2=0.95 


Table  2 .  Values  of  V  when  k  e  0.50. 


BE 

7 

8 

10 

I"!" 

7 

8 

10 

7 

8 

10 

7 

6 

10 

a 

.68 

.61 

.51 

.67 

.  59 

.49 

.65 

.58 

.48 

mm 

.55 

.52 

.47 

.52 

.48 

.44 

.47 

.43 

.39 

HB 

.48 

.45 

.42 

.44 

.42 

.38 

.39 

.37 

.33 

L,  P2 

« 

o 

CD 

A:  P 

L=0 . 2  5 , 

p2=o 

.95 

0 

r> 

o 

it 

P2=1- 

it  04 


Table  3 .  Value*  of  v  when  k  -  l.Q. 


.63 

.56 

.46 

.41 

.38 

.33 

.33 

.31 

.26 

8 

7 

8 

10 

.62 

.55 

.45 

.40 

.36 

.31 

.32 

.29 

.26 

x  *0 . 2  5 , 

V° 

.95 

H 

fl 

1  CM 
CL 

.0 

os  P^"0. 5,  P2»1.0 


Table  4 .  Values  of  V  when  k  *  2.0. 


10 

7 

8 

10 

.47 

.43 

.38 

.28 

.26 

.22 

.23 

.21 

.18 

P2“.y5  A,  P^O.25,  P2=1.0  os  P,“.S,  P2=l.< 


Table  5.  Valuea  of  V  when  k 


8 


8 

10  7 

8  10 

L7 

x: 

.41 

.38 

.33 

.40 

.36 

.31 

.37 

.34 

.29 

As 

.26 

.23 

.20 

.25 

.23 

.19 

.24 

.22 

.18 

O  1 

.20 

.19 

.16 

.15 

.18 

.16 

.23 

.  26 

.14 

x:  P  *0.1,  P  2“0 . 95  As  P^O.25,  P2=1.0  os  P^O.5,  P2=1.0 


Discussion  of  Tables  1.  -  5. 


From  the  entries  of  V  in  Table  1.,  we  notice  little  difference 
in  the  value  of  V  for  P^“0.1  and  P^"0.25  when  P2«0.8  for  all  and 
n^.  Increasing  r.^  results  in  a  reduction  of  V  when  both  P^  and  ?2 
are  large;  thus,  if  we  can  afford  to  increase  n^,  we  should  spread 
both  radii.  Increasing  n^  results  in  a  reduction  of  V  for  all  n^, 

P^  and  p^-  Finally,  for  small  P^(Pj«0.1,  0.2b),  P^O.8  is  better 

than  P  =0.95  or  P  "1.0. 

2  2 

From  Table  2.,  we  see  that  the  smaller  the  radii  P^  and  P^. 
the  larger  the  reduction  in  V  resulting  from  increasing  n^- 
Increasing  r>2  reduces  V  for  all  n^,  Pj  and  P^.  When  both  P^  and 
P2  are  large,  increasing  n^  results  in  a  reduction  of  V. 

When  the  variance  of  the  observations  is  constant  throughout 
the  experimental  region  (Table  3.),  the  value  of  v  decreases  as  we 
increase  0^  and  P Very  little  affect  on  the  value  of  v  as  we 
increase  n^.  However,  as  we  increase  n2,  the  following  reductions 
in  the  value  of  V  occur*  on  a  scale  of  100  percent,  approximately 
6%  reduction  per  one  unit  increase  in  n2  using  x;  approximately 
3%  reduction  per  one  unit  increase  in  n2  using  A,  and,  approximately 
2%  per  one  unit  increase  in  n2  using  o. 

From  Table  4.,  we  see  that  the  smallest  value  of  V  results 
from  using  large  P^^  and  P2>  increasing  n^  has  little  affect  on  v 
except  for  small  P^.  Approximately  a  2%  decrease  in  the  value  of 
V  for  each  increasing  value  of  n2  when  P^"0.25  or  P^*0.50  when 
P2-1.0. 

As  expected,  in  Table  5.,  we  notice  that  P2  should  be  large. 
Little  change  in  the  value  of  V  results  from  increasing  n^  when 
P^»0 . 1  or  P^«*0 . 25 .  However,  when  P^O.5,  the  value  of  V  became 
erratic  When  we  increase  n^  and  n2>  Increasing  n2  results  in  a 
reduction  of  the  value  of  V  except  when  n^*8  and  P^  is  large 
(Pj-,25,  0.50). 
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5.  §unfa»j;y 


In  this  paper,  vs  hsvs  diacusaad  the  problem  of  design  construction 

(using  rotatable  designs)  for  fitting  a  2nd  order  polynomial  in  two 

variables  when  it  is  known  that  the  observations  do  not  possess 

homogeneous  or  constant  variability  throughout  the  experimental  region. 

We  divided  the  region  of  interest  into  two  circular  variance  zones 

where  if  an  observation  is  taken  from  sons  i  (i  ■  I,  n),  it  has 
2  2  2 

variance  (V  and  »  kOj  where  k  takes  one  of  the  values  0.25,  0.5.  1.0. 
2.0,  4.0.  The  case  of  only  two  variance  sones  was  discussed  owing  to 
the  limited  presentation  time.  More  than  two  zones  is  presently  under 
investigation. 

Although  more  extreme  values  of  k  (La,  k»  0.05,  0.1,  10.0)  were 
used  in  addition  to  the  values  above  in  the  investigation  leading  to 
the  writing  of  this  paper,  it  is  hoped  that  the  values  of  k  used  in  thi9 
paper  lend  some  light  on  the  problem  of  design  construction  when  k/  1.0. 
From  the  entries  of  the  value  of  V  presented  in  Tables  1.-5.  ,  we  con¬ 
clude  the  followings  When  k  >  1.0,  and  we  spread  both  design  radii 
to  the  extremes,  we  would  be  using  an  acceptable  design.  However, 
in  assuming  k  -  1.0  and  calculating  the  value  of  v  for  the  above  design, 
we  overestimate  the  value  of  V  by  the  following  amounts t 

when  k  -  20,  using  ■  7,  we  overestimate  v  by  approx.  7% 

r.  •*  8,  •'  "  ’•  "  "  7% 

nl  *  10,  ”  "  "  M  ”  6% 

when  k  •  4.0,  using  =7,  we  overestimate  v  bv  approx.  13% 

n1  ■  8,  ••  "  . .  13% 

nt  -  10,  "  "  . .  15% 

when  k  <  1.0,  the  above  design  (both  radii  spread  to  the  extremes) 
might  be  adequate  when  k  is  close  to  1.0  (i.  e.,  .  4*k<1.0)  and  n^  >  8. 
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However,  when  k  <  .4,  we  -hould  reduce  Pj  to  epproximetely  0.8  and 
el.o  uee  .1*0^. 3.  In  u.ing  the  de.ign  with  0.5  end  p  -  i.0, 

**  the  velue  of  v  by  the  following  quantities. 

when  k  -  0.50.  u.ing  n^  7,  we  undere.timete  v  by  approximately  14% 

nl“  8‘  "  "  -  "  ••  12% 

. .  10% 

whon  k  •  0.25  and  n^  ■  10,  whan  wa  uaa 

n2  •  7.  we  undereatimate  v  by  approximately  30% 

n2'8' .  "  29% 

n2  '  .  •'  28% 
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SYSTEM  PARAMETER  OPTIMIZATION 
USING  RESPONSE  SURFACE  METHODOLOGY 

Mr.  Garry  V.  Barnard 
U.S.  A ray  Missile  Command 


I.  OPTIMIZATION 

This  paper  considers  the  problem  of  determining  the  optimum  set 
of  missile  system  performance  parameters  using  results  from  computer 
simulations.  The  tool  used  for  determining  this  optimum  Is  Response 
Surface  Methodology.  The  concept  of  response  surface  methodology  In¬ 
volves  a  dependent  variable  (Y),  such  as  percent  aircraft  targets  killed, 
and  several  Independent  or  controlled  variables  (Xj ,Xj ,X3 , . . .X.  ) ,  such 
as  radar  detection  range,  single  shot  kill  probability,  reaction  time, 
etc.  If  all  of  these  variables  arc  assumed  measurable,  the  response 
surface  can  be  expressed  as 

Y  -  f(X1,X2,X3,...Xk).  (1) 

The  empirical  response  function,  expressed  in  the  standard  linear 
regression  notation,  Is  given  by 

Y  "  XA  +  L  (2) 

where 

1  Y  Is  an  N  «  1  vector  of  observations  or  responses 

11  X  is  an  N  *  k  matrix  of  constants  which  are  the  preselected  levels 
of  the  Independent  variables 

ill  6_  Is  a  k  »  1  vector  of  unknown  coefficients 

iv  £  Is  an  N  »  l  vector  of  the  error  or  deviations  from  the  true 
response  to  the  estimated  value. 

The  estimator  for  the  unknown  coefficient  vector  £  is  determined  by 
least  squares,  which  is  given  by 

b  -  orxr’x'Y  (3) 

where  is  the  estimated  vector  for  £. 

A  fitted  second  order  response  function,  given  by  equation  (2) 
can  be  written  as 

X  -  bQ  ♦  X'b*  +  (4) 

This  article  has  been  reproduced  photographically  from  the  author's  manuscript. 
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Notice  that  the  JJ'b*  portion  gives  the  first  order  terms  In  the 
response  function  end  the  quadratic  form  X'BX.  gives  the  quadratic 
contribution.  The  stationary  or  critical  point  is  where  the  derivatives 
ai/3X;,  3?/ 3X2 . ..,3^/3X^  are  simultaneously  tero,  which  Is  given  by  the 
vector 

X  -  -l/2B-1b*.  (5) 

The  stationary  point  X  can  be  a  maximum,  a  minimum,  or  a  saddle 
point.  Therefore,  it  will~ Be  necessary  to  determine  the  exact  nature 
of  the  stationary  point.  It  is  convenient  to  express  the  second  order 
response  function  (4)  in  a  different  form  which  can  be  more  clearly 
interpreted  by  the  analyst.  This  involves  translating  the  response 
function  from  the  origin  (Xj  «  0,  X2  "  O,...^  -  0)  to  the  stationary 
point  X  .  Then  the  response  function  can  be  expressed  In  terms  of  new 
variables  V*! ,  W2,...Wk.  Figure  1  illustrates  this  for  two  variables 
where  the  contours  represent  constant  responses. 


*1 

Figure  1.  Canonical  Form  for  a  Response  Surface 
in  Two  Variables. 
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The  function  in  terms  of  the  new  variables  la  callad  the  aWW 
FORM  and  is  given  by 

t  •  f  +  X,Wi2  ♦  X]W22  +  ...  +  XkWk?  (6) 

vhare  Y  la  the  estimated  response  at  the  stationary  point  and  X.'s  are 
the  characteristic  roots  or  eigenvalues  of  the  B  matrix.  The  new 
variables  Wi»W2,...W  are  related  to  the  original  variables  X],X2,...X. 
as  follows  K 

(?) 

where  M  is  a  k  «  k  orthogonal  matrix  the  columns  of  which  are  the 
normalised  eigenvectors  associated  with  the  eigenvalues  X..  Reducing 
the  response  function  (4)  to  the  canonical  form  (6)  Is  called  a  CANONICAL 
ANALYSIS. 

Upon  completing  the  canonical  analysis,  the  analyst  can  more  readily 
interpret  the  response  surface.  By  observing  the  sign  of  the  eigenvalues 
which,  of  course,  are  the  roots  of  the  following  equation 

|  B  -  X!k  |  (8) 

the  nature  of  the  stationary  point  can  be  determined. 

It  can  be  seen  from  the  canonical  form  (6)  that  If  (Xi,X2,,,.x  )  are 
all  negative,  a  move  in  any  direction  from  the  stationary  point  results 
in  a  decrease  in  ?.  Therefore,  the  stationary  point  is  a  point  of  maximum 
response  for  the  fitted  surface.  If  are  all  positive,  X 

represents  a  minimum  for  the  fitted  surface.  For  the  case  where  the  T^s 
differ  in  sign,  the  stationary  point  is  a  saddle  point.  Figure  2  illustrates 
for  two  variables,  a  situation  where  the  stationary  point  is  a  saddle  point. 


Figure  2.  Saddle  Point. 


In  chla  illustration  A j  >  0  and  x2  <  0;  a  nova  along  tha  W2  axis 
away  from  tha  atatlonary  point  In  althar  dlractlon  raaulta  In  a  decrease 
In  estimated  response.  Moving  along  tha  Wj  axla,  a  corresponding 
increase  In  responae  la  obtained.  Tha  same  Indlcatlona  could  be  gained 
In  this  Illustration  by  looking  only  at  tha  canonical  form.  Equation 
(6)  for  the  example  would  be  of  the  fora 

Y  -  Yq  ♦  X,V,2  -  X*V22. 

Clearly,  increases  in  response  would  be  expected  the  smaller  W2  becomes, 
say  taro,  and  the  larger  Wj  becomes.  This  is  equivalent  to  moving  along 
the  W i  axla  away  f roa  X  In  either  direction.  The  point  being  made  here 
la  chat  when  there  are  more  than  two  independent  variables  Involved,  the 
analyst  has  to  rely  on  the  canonical  analysis  rather  than  on  drawings 
of  the  rasponse  surface  to  do  his  analysis. 

Two  axamples  will  be  given  here  to  Illustrate  this  analytical  tool. 

The  data  used  for  chase  examples  are  results  from  sn  sir  defense  simulation 
of  hypothetical  air  defense,  systems  against  a  low  altitude  aircraft  attack 
in  mountainous  terrain.  In  the  first  example,  the  two  parameters  to  be 
optimised  are  the  air  defense  system's  radar  detection  range  and  radar 
tracking  range. 

The  input  values  for  the  two  independent  variables  and  the 
correspond ins  response  (percent  aircraft  kills)  from  the  simulation 
are  given  in  Table  1.  Input  data  for  these  examples  are  coded  auch 
that 


1  the  midpoint  value  of  any  parameter  la  0, 
11  the  smallest  value  of  any  parameter  Is  -1, 
111  the  largest  value  of  any  parameter  is  +1. 


Table  I,  X] ,  X2,  and  Y  for  Example  1. 


Detection  Range 

(Xi) 

Tracking  Range 

<x2> 

Percent  Aircraft  Killed 
<Y> 

-1 

-1 

30 

0 

-1 

50 

1 

-1 

51 

-1 

0 

32 

1 

0 

48 

-1 

1 

32 

0 

1 

44 

1 

^  1 

45 

0 

0 

54 

0 

0 

56 

0 

0 

53 

0 

0 

54 

0 

0 

52 

414 


The  midpoint  for  the  two  variable*  vaa  replicated  five  times  to  set 
an  estimate  of  the  experimental  error.  Using  the  data  given  In  Table 
I,  the  following  response  function 

?  -  *2.93  8.3333X1  -  1.66?X2  -  10.759Xr  -  3.759X2?  -  2.0XiXi 

was  obtained  by  least  squares.  The  analysis  of  variance  in  Table  II 
below  tests  the  significance  of  the  linear  and  quadratic  contributions 
to  the  response  and  the  significance  of  the  lack  of  fit. 

Table  II.  Analysis  of  Variance,  5.xamjrle\  j.X 


Source 

Degrees 
of  Freedom 

Sum  of 
Squares 

T“~  “1 

Lea  ^  ,  •' 

i.qtiarls 

- 1 - 

V 

V-'i  St 

Regression 

5 

968.468 

2i6.wi> 

Linear 

2 

433.333 

/8.4CS* 

Quadratic 

3 

535.135 

ns.i'TR 

81.081* 

Lack  of  Fit 

3 

33.039 

11.013 

£.006 

Error 

4 

8.8 

2.20 

Total 

12 

1010.308 

The  tabulated  F  values  at  the  0.05  significance  level  for  the  degrees 
of  freedom  indicated  in  the  table  vary  from  6.04  to  6.94,  which  indicates 
that  there  is  a  very  strong  linear  contribution,  a  significant  quadratic 
contribution,  and  an  insignificant  lack  of  fit.  This  implies  that  the 
fitted  model  is  adequate,  namely,  that  it  is  not  necessary  to  consider 
higher  order  terms. 

Applying  equation  (5)  results  in  the  following  stationary  point 
DETECTION  RANGE  -  0.418 
TRACKING  RANGE  -  -0.333 
The  eigenvalues  given  by  equation  (8)  are 
*1  •  -3.618  and  ■  -10.899 

which  means  that  the  stationary  point  is  the  point  of  maximum  response. 
The  estimated  response  at  the  stationary  point  is  54.39.  This  system 
is  depicted  in  Figure  3. 

The  drawing  shows  that  tracking  range  Is  the  more  sensitive 
parameter. 
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Figure  3.  Example  1. 

The  next  example  attempts  to  determine  the  optimum  value*  for  cite 
average  missile  velocity  and  radar  tracking  range  for  a  hypothetical 
missile  system  against  the  same  type  threat  and  In  the  same  type  environment 
as  the  first  example.  The  input  data  and  the  results  from  the  simulation 
are  coded  as  before  and  given  in  Table  III. 


Table  III.  Xlt  X:.  and  Y  for  Example  2. 


Average  Velocity 
(Xi) 

Tracking  Range 

(X:) 

Percent  Aircraft  Killed 
(Y) 

-1 

-1 

34 

0 

-1 

38 

1 

-1 

42 

_  1 

X 

0 

50 

1 

0 

60 

-1 

1 

50 

0 

1 

56 

1 

1 

62 

0 

0 

56 

0 

9 

54 

0 

0 

56 

0 

0 

58 

0 

0 

57 

Again  the  midpoint  (Xj  »  U,  X2  •  0)  waa  replicated  five  times  to 
•atlmace  the  experimental  error.  Applying  the  methoda  of  least 
squares  resulted  In  the  following  response  function 

Y  -  56.034  ♦  5. OX)  ♦  9.0X2  -  0.6207Xi2  -  8.6207X22  +  XjX; 

the  analysis  of  variance  performed  on  this  example  Is  given  In  Table  IV. 


Table  IV.  Analysis  of  Variance,  Example  2. 


■  — 

Sourc* 

Degrees 
of  Frttdom 

Sum  of 
Squares 

Mean 

Squares 

F-Test 

Regression 

5 

894.514 

Linear 

2 

635.999 

317.999 

158.99* 

Quadratic 

3 

285.516 

86,171 

39.169 

Lach  of  Fit 

3 

0.993 

0.331 

0.150 

!  Error 

4 

8.8 

2.2 

;  Total 

12 

904. 308 

_ 

— 

The  same  types  of  conclusions  are  reached  with  tills  analysis  of  variance 
as  In  the  previous  example.  The  linear  contribution  is  very  strong,  the 
quadratic  contribution  is  significant,  and  the  lack  of  fit  is  very  small. 

The  stationary  point  for  this  system  is 

AVERAGE  VELOCITY  *  4.666 
TRACKING  RANCE  -  0.7926 

which  Is  far  removed  from  the  experimental  region.  The  eigenvalues  are 
•  -8.6518  and  -  -0.589b 

which  again  indicates  a  maximum;  but  since  the  stationary  point  is 
remote  from  the  region  of  experimentation,  no  valid  conclusions  can 
be  drawn  about  the  surface  around  it.  However,  by  looking  at  equation 
(6)  expressed  for  this  example 

Y  -  Y  -  8.6518L',-  -  0. 5896W-  • 

U 

it  can  be  seen  that  increases  in  response  can  be  obtained  by  V.'-,  equal  to 
zero  and  moving  along  the  W-  axis  toward  the  stationary  point.  This 
case  is  illustrated  in  Figure  4. 
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Figure  4.  Fxample  2. 

In  the  next  section  this  example  will  be  used  for  illustrating  a 
constrained  optimum,  since  the  stationary  point  is  far  removed  from 
the  exneri-enr.il  recion. 

11.  CONSTRAINED  OPTIMIZATION 

If  an  absolute  optimum  cannot  be  obtained  within  the  ranges  of  the 
system  parameters,  it  becomes  necessary  to  determine  a  constrained 
oprinum.  The  usual  mathematical  or  quadratic  programming  techniques 
are  not  used  here  since  they  require  a  concave  function  for  maximization, 
and  this  is  not  always  the  case  in  this  type  of  optimization.  In  fact, 
saddle  points  occur  very  frequently.  A  method  will  be  given  here  for 
determining  a  constrained  optimum  based  on  work  done  by  Hoerl  (1959]  and 
Draper  (1963).  The  essential  requirements  for  this  method  are  (1)  the 
experimental  error  should  be  small,  and  (li)  the  lack  of  fit  should  not 

Kg  t  nn  orgjf  t 

Using  coded  variables  as  described  in  the  previous  section,  local 
optima  will  be  obtained  for  each  sphere  formed  by  varying  radii  R  from 
l':.«  •.“■ur  of  the  experimental  region  (0,0,. ..0)  to  points  (X ; , X  , . , ,X^ ) . 

7  '’0'ecirc  can  easily  be  Illustrated  for  a  two-variable  case  Riven 

1  ••  -  5 . 
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Figure  S.  Constrained  Optimization 
for  Two  Variables. 

The  spheres  for  this  two-variable  case  reduce  to  circles. 

This  method,  by  restricting  itself  to  spheres  of  varying  radii,  finds 
a  local  maximum  on  each  of  the  spheres  (circles  in  Figure  5).  When 
these  restricted  optima  are  determined,  several  two-dimensional  plots 
can  be  drawn  of  Y  against  X),X2,...X.,  the  coordinates  tha  give  rise 
to  these  optima,  and  ?  against  R.  The  analyst  can  then  observe  these 
plots  and  find  the  radius  that  gives  the  maximum  response  and  the 
corresponding  value  of  each  of  the  independent  variables  that  yields 
this  maximum. 

For  each  radius  R,  the  condition  on  the  independent  variables 
(X; ,X?, . . .X,  )  which  will  maximize  Y  subject  to  the  constraint 

£  X12  •  R2  (9) 

L-i 

will  be  determined.  In  matrix  notation,  this  constraint  is  equivalent 
to 

(X'X-R2).  on 

To  maximize  Y  subject  to  the  constraint  (10),  consider  the  function 

F  •  b  +  X'b*  ♦  X'BX  -  c (X'X-R2)  (li> 

c 

where  ■_  is  a  Lagrange  multiplier  and  X.'  ”  ! X]  ,X; , .  .  . X,  ] .  The 
derivative  of  F  with  respect  to  the  vector  >(  is  given  oy 

:-F/3X  •  b*+2BX  -  2„X.  (12) 
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Equating  (12)  to  cero  and  aolvlng  for  X  givea 
(B-uIr)X  -  -l/2b*. 


(13) 


By  selecting  the  proper  value  of  tha  Lagrange  multiplier  u  and 
inserting  in  equation  (13),  the  condition  on  the  independent  variables 
(Xj  ,X2  , . .  .X^)  can  be  determined,  which  will  maximire  Y  for  this  sphere. 
The  radius  of  the  sphere  can  be  determined  from  equation  (10)  and  Y 
from  equation  (4)  of  the  previous  section.  It  should  be  pointed  out 
here  that  this  is  only  the  maximum  on  this  one  particular  sphere  of 
radius  R.  With  several  preselected  values  of  u.  this  procedure  would 
give  local  maxima  on  spheres  of  varying  radii.  Then  plots  as 
suggested  earlier  could  be  given  as  in  Figure  6. 


Figure  6.  Two  Dimensional  Plots. 

The  first  plot  gives  the  radius  R  from  the  center  of  the 
experimental  region  where  the  maximum  response  Is  expected  while 
the  other  plots  show  the  values  of  the  independent  variables  that 
yield  this  maximum.  Not  only  do  these  plots  give  the  X  values  that 
produce  the  maximum  response,  but  the  most  important  result  from  this 
analysis  is  that  a  path  can  be  determined  which,  if  followed,  should 
give  the  quickest  gains  in  response.  Again,  confirmatory  rtr.e  of  the 
simulation  should  be  made  along  this  path  to  insure  that  the  prediction 
is  correct. 

The  second  example  considered  in  the  previous  section  will  be 
used  here  to  illustrate  this  constrained  optimization  procedure. 

Recall  that  in  the  previous  example  the  stationary  point  was  a 
considerable  distance  from  the  experimental  region. 

Table  V  gives  the  results  of  this  analysis. 
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Table  V.  Constrained  Optimisation  Results. 


w 

Average 
Velocity  (Xj) 

Tracking 
Range  (X2) 

R 

Y 

0.5 

2.5121 

0.6311 

2.5902 

68.5098 

1.0 

1.7136 

0.5568 

1.8018 

66.0725 

1.5 

1.2988 

0.5088 

1.3949 

64.4898 

2.0 

1.0441 

0.4728 

1.1462 

63.4004 

2.5 

0.8722 

0.4439 

0.9786 

62 . 6069 

3.0 

0.7484 

0.4194 

0.8579 

62.0010 

3.5 

0.6550 

0.3983 

0.7666 

61.5212 

4.0 

0.5821 

0.3796 

0.6950 

61.1300 

5.0 

0.4757 

0.3478 

0.5893 

60.5256 

7.0 

0.3477 

0.2992 

0.4587 

59.7229 

10.0 

0.2471 

0.2483 

0.3503 

58.9965 

20.0 

0.1251 

0.1594 

0.2026 

57.8859 

The  u' a  In  Table  V  are  the  preselected  Lagrange  multipliers 
and  the  Xj  and  X2  values  were  computed  by  equation  (13)  for  each  u. 

The  R '  s  are  the  square  roots  of  equation  (9)  for  each  X)  and  X2 
combination,  and  the  Y's  are  the  estimated  responses  obtained 
from  equation  (4). 

This  table  gives  a  path  for  which  increases  in  response  should  be 
obtained  more  quickly  by  moving  along  the  path  from  well  within  the 
experimental  region  to  the  outer  fringes  of  the  experimental  region. 
Confirmatory  runs  were  made  of  the  simulation  with  the  first,  second, 
and  fourth  points  of  Table  V  with  the  following  results. 


L\ 

& 

Y 

2, 

.512 

0.6310 

67, 

.57 

1, 

.714 

0.5570 

64, 

.85 

1, 

.044 

0.4728 

61, 

.10 

Each  value  in  the  Y  column  is  the  average  of  five  replications. 
Notice  that  each  response  from  the  simulation  is  slightly  lower  than 
the  predicted  response  in  Table  V.  However,  the  differences  are  v^ry 
small  and  goodly  increases  in  response  are  obtained  by  moving  along 
this  path  as  predicted.  It  should  be  pointed  out  here  that  the  re¬ 
sponse  function  becomes  less  reliable  farther  and  farther  away  from 
the  experimental  region  from  which  the  function  was  based.  If 
further  analysis  1b  desired  outside  the  experimental  region,  a  new 
response  surface  analysis  should  be  performed. 


The  results  of  this  analysis  are  also  plotted  in  Figure  7. 
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Figure  7.  Two  Diaene tonal  Plots,  Example  2. 

From  those  plots  the  maximum  response  can  be  obtained  constrained 
to  the  experimental  region.  Since  f  continues  to  Increase  as  R 
Increases,  the  maximum  response  within  the  experimental  region  should 
correspond  to  the  maximum  R  within  the  experimental  region  which  is 
equal  to  1.414.  The  estimated  maximum  response  at  this  point  is 
approximately  64.0  while  X;  •  1.11  and  X2  ■  0.55  are  the  values  of 
the  independent  variables  (average  missile  velocity  and  radar  tracking 
range)  that  yield  this  constrained  maximum. 

It  was  mentioned  earlier  that  If  the  proper  Lagrange  multipliers 
u  were  selected,  local  maxlmums  could  be  found  on  each  of  the  snheres. 
Meyers  [1970]  shows  that  If  y'a  are  selected  so  that  they  are  greater 
than  the  largest  eigenvalue  X^,  local  maxlmums  will  be  obtained.  On 
the  other  hand,  If  u's  are  chosen  to  thet  they  aro  less  than  che  smallest 
Xj,  local  minimums  will  be  obtained. 

In  this  paper,  only  two  variable  examples  have  been  given  for 
Illustrative  purposes  due  to  ease  of  presentation.  However,  the 
methodology  ie  very  general  and  applies  for  any  number  of  independent 
variables. 
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ABSTRACT  -  A  surrey  La  being  mads  of  work  on  the  biochemistry  and  bicphysios 
of  animal  systems  in  relation  to  tha  specific  talk  performance 
of  auoh  system*. 

Bio-chemioelly  important  systems  which  contain  materials  eueh 
aa  pro  tains,  nucleic  adds,  polysaccharides,  lipoids,  complex  minaral  salts, 
a  to.,  ara  thought  to  ba  rasponsibla  for  particular  animal  functionality 
and  behavior.  This  applies  to  mntelt,  narra,  circulatory  ayatama,  hard 
tissue,  etc, 

Tha  incapacitation  of  animal  systems  (shock)  can  ba  accounted 
for  by  a  modification  of  callular  systems  (enzyme  inactivation,  call  vail 
destruction, ato. )  or  removal  of  tissue  (wound  tract ). 

Consideration  is  being  given  to  tha  uso  of  Biocallular  Numbers, 
which  characterise  tha  absorption,  equipartition  and  conaequenoes  of  energy 
uptake  by  animal  systems  and  which  rsflsct  ths  traumatic  nature  of  the 
reactions  in  terms  of  cellular  biochemistry.... Also  to  protect  against  these 
effect#..  It  is  possible  that  these  numerical  values  can  be  used  Jn  the 
machine  evaluation  of  the  probability  of  occurence  of  oertain  stated  events. 

The  design  of  the  model  should  take  into  consideration  the 
application  of  the  material  mentioned  above  -  in  dimensionality. 
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Multiple  Comparisons  Revisited 


Clifford  J.  Maloney 
Bethe9da,  Maryland 


I.  Introduction :  In  the  years  since  World  War  II,  a  proliferation 
of  approaches  has  arisen  to  the  statistical  analysis  of  data  from  simul¬ 
taneously  oonducted  trials  of  several  rraterials  or  treatments  (3).  The 
existence  of  this  cacophony  of  analyses  strongly  suggests  that  the  true 
key  to  the  situation  has  not  yet  appeared.  In  the  hope  of  discovering 
that  true  key,  a  very  detailed  examination  will  be  made  of  the  applicable 
ideas  in  the  following  section. 

II.  Geometry  of  Multiple  Comparisons:  If  but  a  single  number  is  at 
hand  the  statistician  has  no  contribution  to  make,  though  the  applied  and 
pure  mathematician  can  point  out  respectively  that  (1)  tiie  number  nay 
represent  a  measurement  of  a  particular  property  of  a  particular  object  or 
event  on  the  one  hand  and  (2)  may  be  represented  by  a  certain  segment  on 
the  "number  line"  on  the  other. 

If,  next,  two  numbers  are  available,  the  possibilities  are  enormously 

enlarged.  First,  the  two  nunbers  :ray  be  equal  or  different.  Next, 

whether  equal  or  not  they  ray  correspond  to  (a)  the  same  property  of  the 

same  object  or  event,  (b)  the  same  property  of  different  objects  or  events 

or  (c)  different  properties  of  the  same  or  different  objects  or  events. 

This  article  has  been  reproduced  photographically  from  the  author's  manuscript. 
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In  case  (a)  the  pur*  mathtaatiolan  may  alact  to  represent  than  as  intervals 
on  the  ntxrbar  lira  which  overlap  or  abut.  In  the  latter  oasa,  the  oaibined 
intarval  corresponds  to  ths  sun  of  ths  two  nunbere  (Figura  1).  This  total 
interval  oorrasponda  to  tha  sun  of  tha  two  nunbars  irraspactiva  of  hew  tt>a 
total  is  dividsd  betwaan  then. 


In  aithar  of  oosas  (b)  and  (c)  it  would  be  normal  to  represent  each 
number  on  a  separata  (not  naoassarily  orthogonal)  axis  (Figura  2).  Two 
quite  different  characteristics  of  that  figure  can  be  fixed  for  attention 
but  either  characteristic  applies  to  each  case.  The  sum  of  the  squares  of 
the  two  numbers  gives  the  square  of  the  diagonal  of  the  rectangle  of  which 
the  two  numbers  are  the  sides  and  their  product  gives  its  area.  Here  again 
an  infinity  of  different  pairs  of  numbers  gives  the  same  length  of  df  'gerval 
or  alternatively  the  same  area  though  at  most  two  rectangles  yield  both  a 
given  length  of  diagonal  and  &  given  area. 

By  dividing  the  sum  or  the  sum  of  squares  by  two  or  by  taking  Che 
square  root  of  the  product  we  obtain  the  linear'  mean,  the  quadratic  mean, 
or  the  geometric  mean  of  the  given  two  numbers  respectively.  In  the 
special  case  where  the  two  numbers  are  equal  all  these  means  are  likewise 
equal.  So  far,  this  discussion  has  been  restricted  to  purely  mathematical 
aspects. 

Now  that  we  have  two  numbers  to  consider,  the  statistician  has  free 


scope  for  the  exercise  of  his  talents. 


Of  QOU2*3C  , 


o-f  ‘c  *  C 
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approach  win  vary  depending  on  whether  thasa  two  numbers  are  taken  to 
fall  under  oaaa  (a),  (b)  or  Co).  Wa  have  seen  that,  if  tha  two  numbers 
wars  obtained  under  the  ocndition*  of  oaaa  (a),  i.e. ,  each  is  a  measure 
of  the  same  characteristic  of  a  single  (or  at  least  similar)  object  or 
event,  then  the  statistician  has  the  responsibility  of  explaining  why 
the  two  numbers  differ  (if  they  do).  This  he  does  by  further  separating 
each  of  the  two  numbers  concerned  into  two  parts.  In  the  case  of  the 
linear  or  quadratic  mean  those  two  ocmponents  are  construed  as  entering 
additively  to  yield  the  observed  measurement 

x  *  m  ♦  e.  (1) 

It  is  assumed  that,  so  far  as  tha  model  of  i  igure  1  is  conoemed,  e 
contributes  no  information  and  that  the  "true"  lengths  of  the  two  segments 
are  equal  and  are  equal  to  m.  The  pure  mathenatician  graphs  tha  two  m's 
contributed  by  both  readings  as  intervals  on  a  one-dimensional  axis  since 
each  is  an  exarrple  of  the  same  characteristic.  But  the  statistician  employs 
a  different  condition  to  justify  plotting  the  two  readings  as  abutting 
intervals  on  a  single  axis.  It  is  that  the  two  m's,  being  numerically 
equal,  are  perfectly  correlated  with  each  other.  In  contrast,  it  is 
assum'd  that  the  model  in  Figure  1  is  not  applicable  to  the  two  quantities 
but  that  Figure  2  supplies  the  aorrect  model.  The  e.-  contributed  by 
each  is  plotted  on  a  separate  axis  because  these  values  are  independent 
of  (orthogonal  to)  each  other.  Nevertheless  each  e^  "estimates"  a  true 
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constant  quantity,  a,  which  is  best  estimated  from  the  quadratic  mean, 
i.e. ,  the  diagonal  of  the  rectangle  in  Figure  2.  In  the  case  of  the 
geometric  mean  all  rectangles  are  counted  as  equivalent  which  have  the 
same  area  (not  the  same  diagonal).  If,  in  any  particular  case,  e\  and 
e2  differ  in  magnitude,  this  variation  provides  no  information  about 
the  magnitude  of  o,  whether  estimated  from  the  diagonal  or  from  the 
area  but  o  is  the  sane  for  both  measurements.  Were  it  possible  to 
ascertain  and  plot  o  for  xi  and  x2,  Figure  2  would  take  the  form  of  a 
square  of  .  de  o  and  hence  the  side  estimated  by  the  diagonal  would 
exactly  equal  that  estimated  by  the  area  and  vice  versa.  Since  ei  and 
e2  are  not  in  general  equal,  these  two  estimates  will  not  in  general  be 
either.  But  since,  ay  hypothesis,  the  variation  between  ei  and  e2  is  due 
to  chance,  the  difference  in  the  estimate  of  o  from  the  diagonal  and  from 
the  area  is  likewise  a  result  of  chance. 

So  far  we  have  been  considering  just  two  measurements  under  case  (a), 
but  no  change  in  concept  is  introduced  when  the  number  of  measurements  is 
increased  to  n.  Any  difference  in  magnitude  of  the  several  numbers  is 
attributed  to  the  influence  of  chance.  But  that  chance  itself  is 
characterized  by  a  number,  the  scale  factor  o.  This  quantity  is  estimated 
by  a  mean  just  as  the  linear  rear,  m,  is.  But  th.is  mean  is  a  quadratic 
mean,  sinoe  its  components  are  mutually  independent,  the  same  characteristic 
which  calls  for  a  geometric  representation  as  a  rectangle  in  Figure  2  or  in 
general  an  n-dimensional  rectangular  parallelepiped. 
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Bartlett* 8  test  for  homogeneity  of  variances  elicits  the  contrast 
in  the  two  approaches  of  tlw  quadratic  and  geometric  mean  to  estimates 
of  the  side  of  n-dimeruioml  cubes.  The  length  of  aide  estimated  from 
the  observed  diagonal  length  is  oonpared  with  the  side  length  estimated 
from  the  hyper-valme.  Unless  these  two  magnitudes  are  "acceptably" 
close  the  hypothesis  of  homogeneity,  i.e. ,  that  the  model  of  a  hyper-cube 
applies,  is  rejected.  It  should  be  noted,  however,  that  the  p resumption 
that  a  hyper-cubical  model  applies  to  each  within  class  variance  is  not 
challenged. 

But  the  situation  changes  radically  under  cases  (b)  and  (c).  Now,  if 
our  two  or  more  numbers  differ  we  attribute  those  differences  to  real 
differences  in  the  characteristic  or  characteristics  being  measured  and 
no  chance  effect  is  appealed  to.  Case  (c)  is  beyond  the  scope  of  this 
paper.  Case  (b)  involves  a  situation  in  which,  if  two  measurements  are 
at  hand,  both  are  appxopriately  graphed  cn  the  same  axis,  but  any  differ¬ 
ence  in  their  magnitude  is  attributed  not  to  chance  as  in  equation  (1) 
but  to  a  real  diffeienoe  in  the  two  quantities  of  the  characteristic  being 
measured  in  the  two  instances.  It  is  true  that  the  square  of  their  differ¬ 
ence  is  numerically  equal  to  and  is  even  obtained  by  the  same  arithmetic 
2 

as  the  estimate  of  o  is  obtained  in  case  (a)  which  is  treated  so  fully 
above.  But  the  physical  interpretation  is  entirely  different.  With  two 
readings  and  with  each  reading  interpreted  as  applying  to  a  different 
object  or  event  there  is  no  scope  for  the  action  of  probability  ana  nc 
chanoe  for  the  statistician  to  exercise  his  talents. 
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The  contrast  between  the  statistician’s  approach  to  case  (a)  and 
case  (b)  is  dramatically  evident  as  soon  as  three  measurements  are  at 
hand.  For  new,  in  addition  to  an  estimate  of  c  being  available  under 
case  (a)  the  numerical  algorithm  by  which  it  is  estimated  is  new 
different  from  the  numerical  process  for  estimating  meaningful  differ¬ 
ences  applicable  to  case  Cb).  Further,  in  the  model  of  the  normal 
distribution  for  case  (a)  no  further  parameters  are  introduced  by  this 
third  or  any  subsequent  reading;  whereas  in  case  (b)  every  new  reading 
introduces  K-l  new  parameters ,  the  meaningful  difference  between  its 
value  and  the  K-l  preceding  values.  A  problem  in  pure  mathematics  thus 
arises  in  this  case  that  is  meaningless  in  case  (a).  This  is  the  problem 
of  selecting  a  ’’basis"  for  the  assemblage  of  V2  K  (K-l)  differences 
between  k  measurements.  A  choice  which  will  prove  serviceable  later  is 
set  out  in  Figure  3. 

In  the  figure,  three  values  are  diagrammed  which  may  be  regarded 
either  as  individual  measurements  or  as  means  of  measurements  which  are 
not  regarded  as  differing  only  by  chance  influences  but  as  differing  due 
to  real  influences  of  the  conditions  applying  to  each  measurement.  While 
there  are  three  measurements  or  means,  and  her.ee  three  differences  between 
elements  of  pairs  of  readings  we  may  choose,  as  a  basis  for  this  set  of 
differences,  the  two  abutting  segments  labelled  xi  and  x2  in  the  figure. 

Of  course,  this  model  extends  directly  to  k  readings  or  means,  yielding 
k-l  abutting  intervals  as  a  suitable  basis.  Clearly,  in  the  case  cf  k  means, 
if  any  one  of  the  x,  differs  from  zero,  then  all  values  of  y  to  its  left 
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differ  from  all  values  to  its  right.  These  differences,  x^,  are  of 
course  not  independent,  so  this  choice  is  not  an  orthogonal  decom¬ 
position  of  the  k-1  degrees  of  freedom  for  treatments.  Orthogonality 
is  an  attractive  but  by  no  means  essential  property  of  a  basis  for  a 
set  of  vectors. 

In  short,  the  geometric  representation  of  case  Ca)  is  an  n- 
dimensional  rectangular  parallelepiped,  whereas  the  appropriate 
geometric  model  for  case  Cb)  is  a  set  of  n  points  on  the  number  line, 
each  point  falling  at  that  value  which  corresponds  to  the  Dedekind 
correlate  of  the  number  obtained  by  the  measurement.  Of  oourse,  if  the 
graphing  problem  had  originally  arisen  in  an  analytic  context  devoid  of 
probability  overtones  this  approach  would  be  immediately  adopted  as  it 
is  every  day  in  the  mathematical  classroom.  Again,  for  k  treatments, 
a  basis  for  the  1/2  K  (K-1)  differences  which  they  yield  can  be  found  in 
the  K-1  differences  between  adjacent  measurements. 

III.  Least  Squares  or  Analysis  of  Variance? :  We  use  a  linear  mean 
to  estimate  the  corrmon  component  (m  in  equation  1)  in  a  set  of  measurements, 
but  the  quadratic  mean  to  estimate  the  scale  factor  of  the  random  component 
(e  in  equation  1).  This  is  not  an  arbitrary  choice,  but  is  forced  upon  us. 
The  whole  basis  for  statistics,  for  the  application  of  probability  in 
practical  affairs,  is  that  the  contribution  of  chance  factors  tends  to 
decrease  in  a  mean.  But  a  contribution  is  a  "chance"  contribution  or  a 
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real  difference  depending  on  one’s  point  of  view.  Hie  relation  is  not 
symmetrical.  The  quadratic  mean  has  no  way  of  discriminating  between 
systematic  and  chance  factors,  or  between  chance  components  ascribable 
to  varying  conditions  under  which  the  several  measurements  are  secured, 
but  retains  them  all  indiscriminantly.  The  influence  of  the  constant 
factor,  m,  is  retained  in  the  quadratic  mean,  which  latter  yields  an 
estimate  of  the  scale  factor  only  when  that  of  the  constant  factor  is 
subtracted  out. 

No  problem  of  computation  or  difficulty  in  interpretation  arises 
when  but  one  mean  and/or  one  variance  have  to  be  estimated.  Of  course, 
no  matter  how  many  means  or  parameters  require  estimation,  if  no 
variance  estimate  is  involved,  we  have  a  straight- forward,  if  elaborate, 
problem  of  simultaneous  equations  (at  most).  If  any  number  of  parameters 
are  to  be  estimated,  just  one  variance,  and,  possibly,  the  standard 
error  or  confidence  interval  of  one  or  more  parameters  or  of  linear 
combinations  of  them,  the  appropriate  least  squares  procedures  which  were 
so  highly  developed  in  the  nineteenth  century  apply. 

Hie  usual  multiple  comparisons  situation  fits  this  model  directly. 
There  i '  the  task  therefore  of  accounting  for  the  entanglement  of  this 
problem  with  analysis  of  variance,  if  an  explanation  exists  over  and  above 
the  usual  temptation  to  see  every  problem  as  an  instance  of  the  technique 
most  prestigious  at  the  moment.  The  discussion  of  the  last  section  shows 
how  tliis  temptation  is  at  least  plausible.  A  few  further  remarks  appear 
appropriate . 
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Given  any  set  of  numbers  from  whatever  source,  it  is  arithmetically 
possible  to  calculate  their  quadratic  mean  aid  to  infer  from  it  the 
numerical  value  of  the  scatter  of  the  numbers.  An  appropriate  mean 
value  is  a  useful  quantity  and  that  utility  exists  irrespective  of  the 
significance  or  lack  of  significance  of  the  individual  values  or  of  any 
other  mean  calculated  from  the  numbers.  It  dees  not  however  command  a 
preeminent  position.  Thus,  only  statisticians  would  insist  on  answering 
the  question:  "Is  any  current  governor  of  a  state  of  the  Union  a  wealthy 
man?"  by  first  forming  the  mean  wealth  of  all  governors  and  examining 
that  value.  The  latter  procedure  would  be  appropriate  however  as  an 
answer  to  the  question  whether  possession  of  wealth  is,  on  the  average, 
associated  with  election  as  governor.  That  is,  to  examine  a  question 
about  an  average,  a  long  run  tendency,  we  need  to  form  an  average  and 
examine  it.  To  answer  a  question  about  an  individual,  even  an  individual 
mean,  the  average  is  not  merely  of  no  help,  but  is  more  detrimental  the 
more  values  enter  into  that  comprehensive  mean.  Now,  the  main  effect  for 
treatments  .in  an  analysis  of  variance  is  a  quadratic  mean.  It  answers  the 
question:  "On  the  average,  do  these  treatments  differ  in  effectiveness?” 

It  is  irrelevant  to  the  question:  "Do  a  specified  pair  of  these  treatments 
differ?”  It  is  quite  true  that  if  a  large  number  of  tests  are  made  some 
will  be  erroneous.  That  is  what  the  selection  of  a  significance  level  is 
all  about.  What  is  at  issue  is  whether  or-  not,  and  why,  if  so,  the 
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oombining  of  several  tests  into  one  experiment  should  influence  that 
choice  of  level.  If  an  experimenter  carried  out  trials  in  treatment 
pairs,  one  trial  at  a  time,  presimiably  he  would  not  be  oonplimented  for 
his  efficiency  but,  he  would  be  expected  to  use  normal  t-table  signifi¬ 
cance  levels.  If  he  shared  a  laboratory  with  a  colleague  also  performing 
conparvative  trials,  one  pair  at  a  time,  his  choice  of  tables  would  not 
be  influenced  by  the  presence  or  any  action  of  his  colleague.  The 
freedom  from  influence  would  exist  for  his  colleague  as  well.  Only,  if 
one  of  the  experimenters  conducted  all  of  the  trials ,  and  simultaneously 
at  that,  or  if  both  continued  to  work  but  their  results  were  pooled  would 
the  issue  of  filtering  the  significance  level  arise;  and  then,  in  such  a 
way,  as  to  discourage  experimenters  from  choosing  such  a  combined  experi- 
nental  design,  since  now  any  observed  differences  are  sharply  penalized. 

The  confusion  seems  to  stem  from  viewing  the  F-test  as  a  required 
preliminary  test  which  alone  can  justify  introduction  of  a  multiple 
comparisons  analysis.  But  the  F-test  is  a  test  of  a  mean.  It  is  justified 
and  valuable  where  a  nean  is  called  for;  where  a  small  but  varying  influence 
inheres  separately  in  each  member  of  the  group,  and  where  meaningful  differ¬ 
ences  between  them  are  non-existent  or  are  to  be  ignored  for  the  purposes 
of  the  particular  analysis. 

Since,  in  this  case,  the  numerator  of  F  is  a  quadratic  mean — that  is 
a  variance —  this  mode  of  viewing  the  individual  treatment  means  as  random 


selections  from  a  oontron  pool  for  a  null  hypothesis  was  uncritically 
aiopted.  The  consequence  has  been  confusion,  a  plethora  of  rival 
techniques  and  an  affront  to  ccwnon  sense.  It  may  le  conjectured  that 
the  confusion  would  never  Jrave  arisen  if  the  converse  of  the  principle 
of  the  mean  were  kept  in  view.  The  principle  is  that  all  of  the 
information  about  the  carmen  element  in  a  set  of  readings  is  given  by 
the  mean—and  none  by  differences  between  the  individual  readings.  The 
converse  is  that  no  information  about  intrinsic  (non-chance)  differences 
between  the  individual  readings  is  given  by  the  mean,  which  latter1  are 
the  province  of  comparisons  between  partial  means.  Any  of  these  may  be 
reel  when  the  overall  mean  is  net  and  vice  versa. 

IV.  beast  Squares  Solution:  The  argument  of  Sections  II  and  III 
purports  to  show  that  for  the  solution  of  what  Eisenhart  (2)  has  called 
Model  I,  the  Analysis  of  Variance  is  a  sterile  ctil -de-sac.  This  is 
clearly  revealed  by  the  confused  status  of  the  multiple  comparisons 
problem.  When,  however,  the  problem  is  recognized  as  a  straight-forward 
least  squares  task  in  terms  of  a  natural  choice  of  a  basis  set  for  all 
possible  pairwise  differences  the  difficulty  disap^ars. 

An  example  involving  six  treatment  means  is  diagramed  in  Figure  4. 
The  observed  means  are  denoted  by  y^  and  the  adjacent  differences  by  x^. 
The  figure  illustrates  what  may  have  been  one  source  of  the  failure  to 
recognize  that  the  multiple  comparisons  problem  is  an  instance  of  least 
sauares  in  that ,  what  is  under  study  is  nut  tire  observations  directly 
but  comparisons  between  them.  To  form  observation-,  equations  in  terms  of 
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an  orthogonal  system  of  contrasts  seems  artificial  since  these  are  not 
the  contrasts  of  intei  •  ~n  contrasts  of  interest  are  dependent 

and  it  is  not  usual  tc  -  v, dent  observational  equations.  There 

would  still  be  the  problem  of  choosing  which  set  to  use.  Figure  5  shows 
the  equations  which  result  if  all  possible  (hence  dependent)  differences 
are  used.  There  is  one  fi irthar  difference  between  this  set  and  the  usual 
set  of  observational  equations.  Any  k-1  set  (in  our  case  5)  yields 
which  satisfy  exactly  all  other  equations  in  Figure  5. 

Our  chief  concern  however  lies  not  in  the  estimates  of  the  differences 
but  in  tests  of  their  significance  and/or  in  confidence  limits  for  them. 

For  this  purpose  we  nay  proceed  with  a  conventional  least  squares  solution 
having  regard  for  the  lack  of  independence  in  the  equation  of  Figure  5. 

We  obtain  the  nomal  equations  of  Figure  6.  The  solutions  for  the  are 
given  in  Figure  7.  While  these  results  are  "obvious"  our  purpose  is*  by 
connecting  the  work  with  the  general  least  squares  approach,  to  establish 
that  the  procedure  of  that  method  also  yields  (1)  the  variance  of  every  one 
of  the  */j  K  (K-1)  difference,  (2)  tests  of  significance  for  each,  and  (3) 
confidence  regions  for  each. 

Formally,  or  directly  by  sight  it  is  clear  that  the  variance  of  each 
difference  in  Figure  7  is  2cJ.  Any  chosen  difference  can  be  formed  as  a 
linear  combination  of  x^'s.  Remembering  that  these  are  not  independent 
the  usual  formula  for  the  variance  of  a  sum  yields  the  same  quantity,  2oJ, 


for  each  difference. 


In  vising  Figure  4  as  a  guide  in  our  reasoning  we  have  ignored  the 
fact  that  the  observed  order  of  the  means,  y^,  nay  well  differ  fran  the 
order  of  the  population  values  which  they  estimate.  But  whatever  order 
is  employed,  the  value  2o*  is  obtained  as  the  variance  of  any  differ¬ 
ence.  The  true  order  must  be  included  in  the  set  of  all  possible  orders. 
Hence,  the  result  is  established. 

V.  A  Related  Problem:  The  above  treatment  of  the  multiple  com¬ 
parisons  problem  historically  grew  out  of  a  related  but  distinct 
problem  (1).  When  an  animal  is  exposed  to  a  particular  disease  agent 
it  will  in  general  develop  antibodies  against  a  subsequent  attack.  These 
antibodies  are  highly  but  not  completely  specific  so  that  while  protection 
against  other  attacks  of  the  identical  organism  are  most  completely 
prevented  Cat  a  fixed  level  of  challenge  strength)  a  greater  or  less 
level  of  protection  is  also  afforded  against  closely  related  organisms. 

By  sensitizing  test  animals  with  several  related  organisms  and  then 
challenging  each  animal  with  equivalent  doses  of  each  antigen  complex  a 
matrix  of  responses  is  obtained,  some  of  which  are  homologous  (same 
challenge  as  sensitizing  organism)  and  some  heterologous.  In  the  paper 
referenced,  the  least  square  technique  is  applied  to  ascertain  the 
relative  strength  of  ccrmon  antigens  contained  in  the  several  challenge 
suspensions. 


3 


i 


1 

1 


437 


... 


REFERENCES 


1.  Chaparas,  Sotiros  D. ,  Malanoy,  Clifford  J. ,  And  Hedrick,  Sally  R. 

(1970).  Specificity  of  Tuberculins  and  Antigens  from  Various 
Species  of  Itycrobacteria,  Anar.*  can  Review  of  Respiratory 
Disease ,  Vol.  101,  pp.  74-83. 

2.  Eisenhart,  Qutrchill  (1947).  Die  Assvm*  tions  Underlying  the  Arwlysi 

of  Variance,  Biometrics,  Vol.  3,  No.  1,  pp.  1-21. 

3.  Miller,  Rupert  6. ,  ,Tr.  (1966).  Simultaneous  Statistioal  Infarenoe, 

McGrav-Hill  Book  Ocupany.  " 


XI 


X| 


Figure  1.  Uhivariate  representation  of  nutter*  as  segaants  on  a  single  axis. 
The  svro  of  any  n\jjter  of  segaants  is  a  representation  of  the  son  of  the  roster*. 


Xt  I... .......  -  (X|»X*) 


Figure  2.  Orthogonal  representation  of  two  rosters  as  sides  of  a  rectangle. 
The  diagonal  of  the  rectangle  is  their  quadratic  mean.  The  side  of  a  square 
of  equal  area  is  their  geometric  mean. 


Figure  3.  A  linear  plot  of  the  magnitudes  of  three  measurements  (or  means)  of 
three  different  treatments,  objects,  or  events.  The  two  differences  between 
adjacent  end  points  ere  xi,  and  xi,  the  third  is  their  sveu 


r 


Figure  4.  A  linear  plot  of  six  treatment  moans,  y^. 
adjacent  differences  X£  ora  shew*. 


Xj  ♦  x2  ♦  x»  ♦  x*  ♦  x»  *  y*  -  yi 

Xi  ♦  x»  ♦  x*  ♦  xs  *  y»  -  y2 

xj  ♦  x.  +  x*  *  y»  -  yj 
x»  ♦  xj  *  y*  -  y* 
xt  =  y*  -  y» 

x»  ♦  x*  ♦  x»  ♦  x%  =  ys  -  yi 

x,  ♦  x>  ♦  x,  *  y»  -  y2 

xi  ♦  x*  =  y»  •  y2 
x*  *  y»  -  y* 

X»  *  xa  ♦  Xt  *  y*  -  yi 

x2  ♦  x»  *  y*  -  y2 


xi  *  y*  -  yi 

x»  *  x2  s  y>  -  yi 

x2  *  y»  -  y2 

xi  =  y2  -  y» 


Figure  S.  Observational  equations  for  the  five  parw 
of  Figure  4. 
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The  resulting  five 


values  Xj 


5X1  +  4xt  ♦  3xi  ♦  2x*  +  X|  a  y*  ♦  y$  +  y*  +  y>  ♦  ya  -  Syi 

4xi  ♦  8xi  ♦  6xi  ♦  4x*  ♦  2x#  *  2(y*  ♦  y#  +  y*  ♦  y5)  -  4(yi  ♦  y*) 

3xi  +  6x*  ♦  9xj  ♦  6x*  +  3x»  =  3(y*  +  y$  ♦  y%)  -  3(y»  ♦  y*  +  yj) 

2xi  ♦  4x*  ♦  6xj  ♦  8x%  ♦  4xs  »  4(y»  ♦  v*)  -  2Cy  1  ♦  y*  ♦  yj  ♦  y«») 

xi  +  2xi  t  3xj  +  4x%  ♦  Sxs  s  5y§  -  (yi  ♦  y»  ♦  y*  ♦  y*  ♦  ys) 

Figure  6.  Normal  equations  for  the  parameters  xi  formed  from 
tiie  observational  equations  of  Figure  5 . 


xs  s  y«  -  y* 
xh  =  y$  -  y* 

Xj  a  y%  -  ys 

x*  *  yj  -  y2 
xi  *  yj  *  yi 

Figure  7.  Solutions  of  equations  (6)  for  parameters  Xj . 
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AN  OPTIMAL  U  (MIN)  DESIGN  FOR  ESTIMATING 
THE  SLOPE  OFA  SECOND  ORDER  LINEAR  MODEL 


L.  Otc  and  W.  Mandanhall 
Department  of  Statletlca 
University  of  Florida 
Gainesville,  Florida 


The  problem  presented  here  concerns  estimation  of  the  elope  of 
a  second  order  linear  model.  Previous  results  are  used  to  obtain 
the  experimental  design  vhlch  minimises  s  linear  cost  function  sub¬ 
ject  to  the  restriction  that  the  variance  of  the  estimated  slope 
Is  constant  for  a  given  setting  of  the  Independent  variable. 

1.1  Introduction 

Experimenters  in  chemistry,  biology,  and  engineering  are  frequently 
concerned  with  the  examination  of  second  order  response  models. 

The  objective  of  such  studies  may  involve  estimation  of  the  rates  of 
change  of  the  response  for  given  values  of  the  Independent  variable. 

For  example,  a  chemist  In  the  pharmaceutical  industty  might  want  to 
examine  the  ratea  of  change  in  the  response  of  rats  to  different 
doses  of  a  drug  product.  Or,  an  engineer  might  be  Interested  In 
studying  velocities  of  a  vehicle  using  a  particular  solid  fuel  pro¬ 
pellant  , 

In  this  paper  we  will  assume  that  the  slope  of  the  second  order 
response  curve  Is  estimated  using  the  method  of  least  squares.  We 
then  consider  the  experimental  design  which  minimizes  a  linear  cost 
function  sublect  to  the  restriction  that  the  variance  of  the  least 
squa.e  estimator  of  the  slope  at  a  given  setting  of  the  independent 
variable  Is  equal  to  a  fixed  value,  B. 

1.2  Background 

We  will  consider  the  following  second  order  model  relating  a 
response,  y,  to  an  independent  variable,  x. 

2 

y  ’  30  +  31X  +  a2x  +  €  (D 


The  remainder  of  this  article  has  been  reproduced  photographical ly  from 
the  author's  manuscript. 
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A  three-point  experimental  design  is  to  be  used  with  design  points 
x  C  x2  <  xr  We  will  also  allocate  n^  observations  to  design  point 
x  (i  -  1,2,3)  where 

i-1 

It  is  convenient  to  code  the  independent  variable,  x,  about 
the  midpoint  of  the  design  region.  Let 


Hence  the  design  points  for  the  new  independent  variable,  u,  can  be 
labeled  -1,  u  ,  and  1  and  equation  (1)  can  be  rewritten  as 

y  -  eQ  +  BjU  ♦  82u2  +  c  (3) 

The  estimator  of  the  slope  for  a  given  value  of  the  independent 
variable,  u,  is  ' 


s|u  -  9X  +  2s2u 


(4) 


and  the  variance  of  s  u  is 


2..,; 


V (s | u)  -  V(0L)  +  4u  V(B2)  +  2u  Cov(Pl,i2) 


(5) 


Equation  (5)  traces  a  parabola  as  a  function  of  u  and  can  be  re¬ 
written  as 


V(s|u)  ■  o2C(u  -  h)2  +  q2k 


(6) 


uuu 


see  (Ott  and  Mendenhall,  1970) .  The  coordinates  of  the  vertex  of 
the  variance  parabola,  (h,  k) ,  can  be  shown  to  be  the  expressions 

h  -  -V1  -  Ul1(bU2  •  •>  ,  (7) 

2W 


4nin3 


n2(bu2  -  a)' 
4Wn^n3 


with  C  -  - - - <9> 

nin2n3  ^1  “  U2* 

where  a  *  -  n^,  b  »  n^  +  n^  and 

2  2  2 
W  -  n2bu2  "  2n2aU2  +  n2b  +  b  ~  a 

V(slu) 

We  have  illustrated  u  for  the  design  a  =  0,  n  =  n  *  8 

o 

and  Hj  *  4  in  Figure  1. 

Examining  equation  (6)  we  see  that  the  variance  parabola,  v(s|u) 

is  determined  by  the  location  of  its  vertex,  ;  k) ,  and  the  rate  of 

2 

curvature,  C,  Changes  in  the  quantity,  a  ,  merely  shift  the  entire 
parabola  upward  or  downward. 

From  a  practical  standpoint,  we  would  like  the  vertex  of  the 
variance  parabola  to  be  near  the  center  of  the  experimental  region. 
That  is,  we  wish  h  to  be  near  (or  equal  to)  zero.  We  would  also  like 
k,  the  variance  at  the  vertex  of  the  parabola,  to  be  small.  Ott  and 
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Mendenhall  (1970)  showed  that  the  interior  design  point,  u2(min), 
which  minimizes  the  variance  at  the  vertex  of  the  parabola  is  given 
by 

u2(»in)  •  £  . 

It  can  be  shown  that  h  »  0  for  all  u2(min)  designs  (designs  of  the 
form  -1,  1).  Hence  for  a  given  sample  size  and  allocation  (n. , 

w  1 

n^,  and  n^)  a  u^tmin)  design  selects  a  value  for  the  interior  design 
point,  u^,  which  not  only  minimizes  the  variance  of  the  vertex  of  the 
variance  parabola  but  also  locates  the  vertex,  (h,  k) ,  in  the  center  of 
the  experimental  region.  The  classification  of  (min)  designs  can 
be  refined  even  further.  Ott  and  Mendenhall  (1970)  showed  that  a 
symmetrical  u2(min)  design  (a  *  n3  -  n^  *  0)  is  to  be  preferred  to  a 
general  u2(min)  design  since  both  k  and  c  increase  as  the  design  loses 
its  symmetry.  We  shall  restrict  our  attention  to  symmetrical  u2 (min) 
designs  in  the  remainder  of  this  paper, 

1.3  Minimizing  the  Cost  of  Experimentation 

An  optimal  experimental  design  is  one  that  purchases  a  specified 
quantity  of  information  at  minimum  cost.  We  shall  determine  the 
symmetrical  u2(min)  design  which  minimizes  the  cost  of  sampling  sub¬ 
ject  to  the  restriction  that  V(sjx)  equals  a  specified  value,  B. 

We  will  assume  a  cost  function  of  the  form 

Cost  =  +  n2C2  +  r‘3Cjt  (10) 

where  is  the  cost  of  obtaining  an  observation  at  the  i-th  design 
point  (i  =  1,2,3).  For  symmetrical  u2(min)  designs  we  rewrite  equa¬ 
tion  (10)  as 

Cost  =  n^d^  +  n2d2  (11) 
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with  d,  =■  2c,  and  d  *  c_. 

11  2  2 

We  define  a  function,  I,  which  will  be  used  to  obtain  the  sym¬ 
metrical  u^fmin)  design  that  minimizes  equation  (11)  subject  to  the 
restriction  that  v(s|x)  *  B. 


*  “  nldl  +  n2d2  +  ^CV(s|x)  -  B] 


(12) 


Substituting  into  equation  (6)  with  n3  -  we  find 


V(s  |  x)  .  I*  -  9)‘  *  a\ 


(13) 


where 


C  = 


2n 


nin2 


and  k  = 


2n. 


Hence 


.  2  .  .  2  .  2  ,  2  2 
♦  .  Vl  X  +  . 

r  n  “  - 


r  n. 


J  (14) 


We  obtain  a  system  of  three  equations  in  the  three  unknowns 
(n^,  n^  and  X)  by  taking  partial  derivatives  of  }  with  respect  to 
these  unknowns. 


,rlz2l*-.ZL' 
v  .  2  2 
r  ni 


0 


(15) 


,  2,  .  2 

r4j  (x-g)  ",  _ 

*  2  2  J  " 

r  n2 


0 


11 

ax 


4?2(x-e)2  2?2 (x-0) 2 

2  2 


r  n. 


r  n. 


B 


0 


(16) 


0-7) 
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Solving  equations  (IS),  (16)  and  (17),  wo  obtain 


ni  “  f“  (2D  +  1/2)f^  +  x] 


(18) 


and 


n2  =  4n 


(2D  +  1/2)1 


where  D 


(x  .-.ai: 
2 
r 


(19) 


It  can  be  easily  shown  that  the  results  presented  in  equations 
(18)  and  (19)  provide  the  sample  size  (2n^  +  n^)  and  allocation  of 
the  sample  size  (n^,  n2#  n^)  for  a  symmetrical  u2 (min)  design  which 
minimize  the  cost  function 


Cost  -  +  n2d? 


(20) 


subject  to  the  restriction  that  the  variance  of  the  estimated  slope 
at  a  point,  x,  equals  B.  Note  the  solution  is  location  invariant. 
If  d^  *  d^,  i.e.,  the  costr  are  equal,  the  optimal  solution  is 


n 


1 


(2D  +  1/2) 


and  n2 


4n^D  • 

(2D  +  1/2) 


1 . 4  Example 

Consider  the  following  example.  We  wiBh  to  estimate  the  slope 

of  a  second  order  model  where  the  range  of  tho  experimental  region 

2 

is  r  ■  1.0  and  the  variance  of  the  response,  y,  is  a  *1.0.  We 
t.'ill  assume  that  the  experimenter  is  interested  in  estimating  the 
slope  at  a  distance  of  .5  from  the  midpoint  of  the  experimental  region 
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so  that  the  variance  of  the  estimated  slope  at  this  distance  is 
B  ■  .1.  Further  assume  that  it  costs  d.  «  $1.00  to  obtain  an  obser- 

X 

vation  at  either  of  the  exterior  design  points  and  d2  *  $2.00  at 
the  interior  point. 

2  2 

We  use  equation  (21)  with  (x  -  9)  ■  (.5)  and  D  -  .25  to  obtain 


[©  * 


Substituting  into  equation  (19)  we  have 


n2  '  “K»)-27 


TtfriTJj) 


The  symmetrical  u2 (min)  design  (in  units  of  u)  is  then  n^  =  30, 
n2  *  15,  and  *  30  observations  at  the  design  points  -1,  0,  and  1 

respectively,  we  can  easily  convert  this  to  units  of  x  using  the 
relationship 


x  -  9 


Thus  if  the  midpoint  of  the  experimental  region  for  this  examp.e  is 
9  =  5,  we  obtain  the  design  points  4,  5,  6  with  allocations  30,  15 
and  30,  respectively. 
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1 , 5  Summary 

in  this  paper  we  have  discussed  the  use  of  u^min)  designs 
for  estimating  the  slope  of  a  second  order  linear  model.  We  then 
determined  the  symmetrical  u2(min)  design  which  minimizes  a  cost 
function  of  the  form 

Cost  «  ni_di  +  n2d2 

subject  to  the  restriction  that  V(s|x)  ■  B. 
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THE  ANALYSIS  OF  COMPLEX  CONTINGENCY 
TABLE  SATA  FROM  GENERAL  EXPERIMENTAL  DESIGNS 
AND  SAMPLE  SURVEYS 

Gar?  G,  Koch  and  Donald  W.  Ralnfurt 
Department  of  Bloatatlattca 
University  of  North  Carolina  at  Chapal  Hill 
Chapel  Hill,  North  Carolina 

SUMMARY 

Initially,  a  number  of  asymptotically  equivalent  approaches  to  the 
analysis  of  complex  contingency  table  data  are  outlined  and  contrasted. 
Those  Include  maximum  likelihood,  minimum  discrimination  information,  and 
the  general  linear  model  procedures.  Various  test  criteria  are  presented 
along  with  a  number  of  methods  for  sstimetlon.  Dus  to  the  generellty  of 
the  linear  model  approach  and  its  familiarity  from  the  contmuoua  cate,  it 
ie  the  procedure  chosen  for  the  sequel. 

The  notetlon,  assumptions,  and  methods  of  Inference  for  the  general 
linear  model  approech  are  detailed  as  in  Gristle,  Starmer,  and  Koch  [1969] 
and  are  illustrated  for  the  familiar  teat  of  independence  for  tvo-vay 
contingency  tables. 

A  more  general  categorical  data  model  is  developed  Cor  the  situation 
in  which  the  data  are  Incomplete  in  the  sense  that  not  all  of  the  experi¬ 
mental  units  are  classified  according  to  each  of  the  dimensions  of  the 
table.  Since  tbit  information  relative  to  a  subset  of  the  dimensions  of 
the  table  is  assumed  to  arise  by  design  rather  than  chance,  it  is 
referred  to  as  "supplements!  Information"  and  the  relevant  margins  of  the 
table  as  "supplemented  margins."  This  supplemental  information  Is  used 
to  Improve  the  precision  of  the  estimates  of  the  marginal  probabilities 
over  the  supplemented  dimension(s) ,  and  it  is  Incorporated  into  the  esti¬ 
mation  of  the  individual  cell  probabilities  in  order  to  Improve  certain 
tests  of  hypotheses,  especially  those  involving  marginal  symmetry.  Note 
is  made  of  rhe  obvious  resemblance  to  double  sampling. 

Special  notation  for  tha  case  of  supplemental  data  Is  developed  and 
a  weighted  least  squares  estimation  procedure  outlined,  A  general  two- 
stage  test  procedure  resembling  that  for  the  unsupplemented  case  is 
indicated . 


Contingency  table  date  often  arlee  fro*  eaaple  aurveye  of  finite 
populations.  It  la  noted  herein  that,  with  certain  aodlf icatlona  conelatent 
vith  the  principles  of  survey  sampling  (via,  modifications  to  the  eatiaate 
of  the  covariance  aatrix),  the  aore  general  linear  model  approach  applies 
directly. 

Finally,  a  number  of  detailed  examples  are  presented.  These  exaaplaa 
llluetrate  the  general  linear  aodel  approach  to  categorical  data  arlalng 
In  the  following  contexts t 

(1)  a  simple  quant nl  biological  assay 

(2)  a  factorial  design-type  contingency  table  Including  interaction 
(Dyke  and  Patteraon,  [1952}) 

(3)  a  split-plot  contingency  table  on  social  mobility  trends  for 
different  countries  (hosteller,  [1968]) 

(4)  a  comparison  of  two  drugs  where  there  is  suppleaental  information 
for  both  drugs 

(5)  a  sample  survey  situation  involving  questionnaires  sent  to  a 
sample  of  newly- 11 censed  drivers  in  North  Carolina 

1.  SOME  HISTORICAL  REMARKS 


1.1  Introduction 

Categorical  data  and  its  analysis  have  been  of  interest  to  statis¬ 
ticians  ever  since  the  earliest  origins  of  the  subject.  For  example,  in 
the  area  of  vital  statistics,  Graunt  presented  data  during  the  early 
1600's  on  causes  of  death  (£.£. ,  consumption,  diseases  of  Infancy,  tooth 
diseases)  which  appeared  In  tK.  '  irm  of  frequency  tabulations  according  to 
the  sex  and  marital  status  of  tr.e-  deceased.  Since  1790,  census  studies 
have  collected  and  analysed  numerous  cross-tabulations  on  a  wide  variety 
of  demographic  variables.  Another  example  is  in  genetics  where  Mendel 
collected  data  in  the  mld-1800's  on  garden  pea  varieties  (one-way  tables) 
leading  to  the  now-famous  Mendellan  heritablllty  ratios.  Finally,  In  the 
field  of  epidemiology.  Greenwood  and  Yule  in  the  early  1900's  had  considerable 
data  on  typhoid  attacks  on  inoculated  and  non-lnoculated  individuals. 


The  remainder  of  this  article  has  been  reproduced  photographically  from 
the  author's  manuscript. 
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Aside  from  this  hlstorlcel  background,  categorical  data  appearing 
in  the  form  of  frequency  tabulations  In  one-way.  two-way,  and  multi-way 
tables  are  a  familiar  data  array  to  virtually  all  statisticians  In  today's 
society.  The  specific  examples  range  from  experiments  In  medicine  dealing 
with  all  or  nothing  responses  to  various  stimuli  or  methods  of  therapy,  to 
investigations  of  highway  safety  data  indicative  of  degree  of  personal 
injury  severity  in  motor  vehicle  accidents,  and  to  sample  surveys  dealing 
with  various  aspects  of  life  Including  even  such  personal  matters  as 
family  planning.  In  most  of  these  situations  which  cut  across  the  soc.al 
sciences,  the  life  sciences,  and  even  the  physical  sciences,  the  data  are 
expressed  In  terms  of  nominal  or  ordinal  scales.  Moreover,  oven  quanti¬ 
tative  data  are  occasionally  represented  in  a  categorical  form  when 
grouped  Into  class  intervals. 

Although  many  statisticians  are  aware  cf  the  various  Interrelation¬ 
ships  that  may  exist  among  the  classification  type  variables  in  a 
multl-dlmensional  contingency  tabic,  such  data  are  often  analyzed  in  terms 
of  the  traditional  Pearson  chi-square  statistic  as  applied  to  some  subset 
of  the  totality  of  two-dimensional  tables.  Since  these  statistics  are 
directed  In  many  cases  at  fragmented  hypotheses,  a  definite  need  becomes 
apparent  for  comprehensive  methods  of  statistical  analysis  which  are  directly 
applicable  to  qualitative  data  and  which  are  analogous  in  scope  and  power  to 
multiple  regression  and  multivariate  analysis  of  variance  as  applied  to 
quantitative  data.  Unfortunately,  because  categorical  data  has  been  of 
Interest  to  a  wide  variety  of  researchers  with  divergent  areas  of  application, 
only  a  limited  amount  of  attention  has  beer  directed  at  general  models 
and  methods  of  inference  in  the  published  literature.  Instead,  more 
consideration  has  been  given  to  the  development  of  Miscellaneous  techniques 

which  are  discussed  from  somewhat  different  philosophical  points  of 
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vi«w  and  which  ara  applicable  to  a  aeries  of  not  alvaya  well-defined 
apacial  caaaa .  Thus,  tha  praotitionar  haa  of tan  had  to  aaarch  the 
literature  for  a  procedure  appropriate  to  hie  problem;  when  he  has 
failed  to  find  auoh  a  technique,  he  haa  either  been  required  to  uaa  an 
Inappropriate  analyaia  for  hia  data  or  leave  his  data  un&nalysed. 

Another  difficulty  in  the  analyaia  of  categorical  data  haa  been 
the  existence  of  numerous  methods  of  estimation  and  corresponding  teat 
oritaria  upon  which  statistical  Inferences  may  be  baaed.  Aa  a  result, 
even  whan  the  researcher  nss  decided  upon  the  hypotheses  of  interest, 
he  still  is  confronted  with  a  wide  choice  of  particular  methods  to 
apply.  In  most  cases,  this  choice  is  not  crucial  since  many  of  the 
methods  have  been  shown  to  be  asymptotically  equivalent.  However, 
since  various  authors  use  different  methods,  the  variety  of  choices  is 
often  confusing  to  the  uninitiated. 

1.2.  Some  Recent  Developments  in  General  Methods  of  Analysis  for 
Categorical  beta 

Pearson  (1947]  was  among  the  first  to  note  that,  even  in  the 
simple  2x2  contingency  cable,  Che  same  configuration  could  have  arisen 
from  different  sampling  schemes.  Thus,  it  appeared  necessary  to  care¬ 
fully  specify  the  underlying  probability  model  since  different  models 
could  lead  to  different  statistical  procedures  with  obviously  different 
interpretations . 

The  importance  of  model  construction  and  hypothesis  formulation 
In  complex  multi-dimensional  contingency  tables  has  been  discussed  in 
some  detail  by  Bhapkar  and  Koch  [1968a,  1968b].  Their  work  emphasized 
the  relationship  between  certain  problems  in  categorical  data  situations 
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with  corresponding  problems  in  multivariate  analysis  pertaining  to  pat¬ 
terns  of  correlations  end  factorial  analysis  of  ef facta.  As  such ,  it  wee 
a  direct  outgrowth  of  earlier  work  pioneered  by  Roy  end  Mitre  {1956],  Roy 
end  Kaatenbaua  [1956],  Roy  and  Bhapkar  [1960],  end  Bhapkar  [1961,  1966, 
1968] .  Underscoring  the  results  of  this  research  has  been  the  careful 
consideration  of  the  sampling  • chans  froa  which  the  date  arose.  The 
particular  factor-response  structure  (i.e. .  sampling  scheme)  of  the  model 
has  dictated  the  hypotheses  of  relevance  Just  as  in  univariate  and  multi¬ 
variate  analysis  of  variance  for  continuous  variables. 

The  application  of  this  approach  to  data  from  a  variety  of  experi¬ 
mental  situations  through  the  use  of  linear  regression  models  has  been 
described  by  Gristle,  Scanner,  and  Koch  [1969],  Gristle  and  Williams 
[1570],  Johnson  and  Koch  [1970a],  Koch  and  Relnfurt  [1970],  and  Reinfurt 
[1970].  These  authors  base  their  analyses  on  tha  use  of  a  fairly  power¬ 
ful  and  widely  applicable  computer  program  which  has  been  documented  by 
Forthofer,  Grizzle,  and  Sterner  [1969].  The  resulting  test  statistics 
are  derived  through  weighted  least  squares  procedures  and  correspond 
identically  to  the  modified  minimum  chi-square  statistics  (xj)  due  to 
Neyman  [1949]  or  equivalently  the  generalized  quadratic  form  criteria 
due  to  Wald  [1943]. 

Another  line  of  development  has  been  pursued  by  researchers  who 
are  particularly  Interested  In  multiplicative  models  for  multinomial 
populations.  Here  a  maximum  likelihood  approach  based  on  iterative  pro¬ 
portional  fitting  of  appropriate  marginal  sub-tables  has  been  emphasized 
A  comprehensive  discussion  of  this  approach  has  been  given  by  Goodman 
[1969,  1970].  These  papers  bring  together  a  number  of  results  obtained 
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previous.lv  b  •  ;ji  rch  >  >  •  ..  '  .adman  [1063/1,  1963b,  1964a,  1964b], 

Mosteller  [1968],  Bishop  [1969],  rienberg  [1969]  and  Plackett  [1969]. 

A  substantial  part  of  this  effort  has  been  motivated  by  the  resemblance 
of  multi-dimensional  contingency  tables  to  certain  complex  factorial 
designs.  Maximum  likelihood  methods  are  then  applied  to  the  assumed 
multiplicative  underlying  models.  Finally,  Bishop  and  Fienberg  [1969] 
and  Goodman  [1968]  have  demonstrated  that  this  approach  is  a  particularly 
useful  one  for  handling  incomplete  contingency  tables  (i.e..  ,  tables 
containing  cells  with  zero  frequencies). 

Finally,  a  third  general  approach  which  bears  a  definite  resem¬ 
blance  to  the  maximum  likelihood  methodology  previously  discussed  is 
based  on  the  principle  of  minimum  discrimination  information.  Certain 
aspects  of  this  type  of  analysis  are  summarized  in  Ku  and  Kullback  [1968]. 

Some  other  relevant  results  are  given  in  Kullback  [1959],  Kullback, 

Kupperman,  and  Ku  [1962],  Good  [1963],  Irel.and  and  Kullback  [1968n,  1968b],  and 

Ireland,  Ku,  and  Kullback  [1969]. 

For  the  most  part,  all  of  these  approaches  are  asymptotically 

equivalent  in  the  sense  of  being  based  on  BAN'  estimates  (see  Section  2.3) 
as  described  by  Neyman  [1949].  Certain  results  of  Hoeffding  [1965] 
and  Rao  [1963]  suggest  that  the  maximum  likelihood  approaches  are  more 
efficient.  However,  the  authors  here  believe  the  minimum  -approach 
to  be  computationally  simpler  and  to  bo  more  robust  against  departures 
from  underlying  model  assumptions.  Hence,  it  will  be  the  one  which  is 
emphasized  in  the  remainder  of  this  paper.  KinaLly,  it  should  be  noted 
that  the  relative  merits  of  the  various  approaches  have  been  discussed 
a  number  of  times  by  Berkson  [1955,  1957.  1968]  in  the  context  of  data 
from  the  health  and  medical  sciences. 
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This  discussion  has  focused  attention  primarily  on  the  three 
general  methods  of  analysis.  Other  important  contributions  which  have  cut 
across  these  lines  of  research  and  supported  their  development  include 
the  work  of  Bartlett  [1935],  Simpson  [1951],  Lancaster  [1951],  Plackett 
[1962],  Lewis  [1962],  and  Darroch  [1962]  on  contingency  table  interaction; 
Cochran  [1952]  and  Maxwell  [1961]  on  methods  which  pertain  to  the  refining 
and  strengthening  of  the  applicability  of  the  traditional  Pearson  type 
X2“test;  and  Berkson  [1944,  1953,  1954,  1968]  and  Grizzle  [1961]  on 
applications  to  data  arising  from  bioassay  models. 

2.  MODELS  FOR  CATEGORICAL  DATA 


2.1.  The  Characterization  of  Multi-dimensional  Contingency  Tables 

In  the  analysis  of  any  multi-dimensional  contingency  table,  there 
are  three  fundamental  aspects  to  be  considered.  These  include 

(i)  the  specification  of  the  underlying  model; 

(ii)  the  formulation  (in  terms  of  this  model)  of  the  hypothesis 
to  be  tested  and  the  calculation  of  the  corresponding  test 
statistic ; 

(iii)  the  interpretation  (with  respect  to  the  model)  of  the 
results . 

In  order  to  specify  the  .derlying  model,  it  must  be  realized  that  each 
subject  (or  experimental  unit)  may  give  rise  to  two  types  of  data. 

These  are 

(i)  a  description  of  the  experimental,  conditions  which  the 
subject  undergoes  (or  of  the  sub-population  of  units  to 
which  tiie  experimental  unit  belongs) — henceforth  referred 
to  as  "factors"  or  "populations”; 

(li)  a  description  of  what  subsequently  happens  to  each  subject  — 
henceforth  referred  to  as  " I'espojises" . 

In  other  words,  the  data  in  any  mu  1  L  i-d  iiiiens  i  ona  1  contingency  [able  are 
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the  frequencies  with  which  subjects  belonging  to  the  same  sub-population 
or  combination  of  factor  categories  yielded  the  same  combination  of 
responses.  The  dimension,  d,  of  a  contingency  table  refers  to  the  total 
number  of  factors  and  responses  while  the  levels  represent  the  sub-categories 
within  the  factors  and  responses. 

More  specifically,  consider  r  independent  random  samples  taken 
from  r  multinomial  populations  where  n^represents  the  size  of  the 
sample  from  the  i-th  population  and  n„  the  observed  number  falling  in 
the  j-tli  category  of  the  i-th  sample  where  i  «■  1,  2,  ...,  r  and 

j  =  1,  2,  ...»  s.  It  should  be  noted  that  i  and/or  j  may  be  multiple 

subscripts  as  in  the  case  of  multi-factor  and/or  multi-response  models. 

For  example, 

i  *  i'  =  (i^ ,  i2,  ....  id  )  with  i^  =  1 ,  2,  . . . ,  r^,  a  =  1,  2,  . . . ,  df 

j  *  j'  =  (jlt  j2»  •••.  jd  )  with  jg  =  1,  2,  sg)  3  =  1,  2,  ...,  dr> 

In  this  context,  the  model  is  that  of  a  df-factor,  d^-response  contin¬ 
gency  table  with  the  factors  at  levels  i^,  a  =  1,  2,  ...,  df.  and  the 

responses  at  levels  j&,  6  =  1,  2,  ...,  df.  Here,  it  will  be  assumed 

that  all  response-level  combinations  occur  with  positive  probability; 
however,  situations  where  this  assumption  does  not  hold  may  be  handled 
by  methods  analogous  to  those  given  by  Goodman  [1968].  On  the  other 
hand,  not  all  factor-level  combinations  are  required  to  appear.  In 
particular,  if  the  sample  design  is  an  incomplete  block  design,  not  all 
factor-level  combinations  appear . 

From  these  consid: rations ,  it  follows  that  four  types  of  multi¬ 
dimensional  contingency  table  models  can  be  identified.  Those  are  as  follows: 
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Modal  I.  No  factor,  multi-response  tab la* 

Modal  II.  Unl-factor,  tnulti-rasponaa  tablaa 
Modal  III.  Multi-factor,  uni-re»pon«a  tables 
Modal  IV.  Multi-factor,  multi-responsa  tables 
Obviously,  for  ona-vay  tables,  only  Nodal  I  can  occur;  for  two-way 
tables,  only  Models  I  and  11  can  occur; for  three-way  tables,  only 
Models  1,  II,  and  III  can  occur.  Otherwise  (d  >  3),  all  four  models 
can  arise  In  various  situations. 

To  identify  the  hypotheses  appropriate  to  the  different  models, 
consider  the  case  of  three-dimensional  tables.  In  the  case  of  "no 
factor,  three  response"  tables  (Model  I),  the  primary  interest  lies  in 
the  relationships  among  the  three  responses  which  are  analogous  to 
problems  of  independence  and  correlation  in  normal  multivariate  analysis 
of  variance.  Questions  of  interest  include  total  independence  of  the 
three  responses,  independence  ot  any  one  response  iron,  the  other  two, 
pairwise  Independence,  and  partial  association  between  two  responses 
withi.i  given  levels  of  the  third.  In  addition,  hypotheses  of  total 
symmetry  end  marginal  symmetry  are  often  of  considerable  interest. 

If  the  experiment  is  of  the  "one  factor,  two  response"  type 
(Model  II),  interest  lies  in  the  association  between  the  responses  as 
well  as  the  effect  of  the  factor  on  the  responses.  Here  the  questions 
posed  are  similar  to  those  encountered  in  one-way  normal  multivariate 
analysis  of  variance.  Appropriate  hypotheses  for  this  case  include 
Independence  of  the  responses  within  each  factor  level,  homogeneity  for 
each  marginal  response  (i.e.,  does  the  factor  level  affect  the  marginal 
distribution  of  the  response?),  and  j.iint  homogeneity  O.e.  ,  docs  the 


factor  level  affect  the  Joint  distribution  of  the  responses?)  . 

Finally,  if  the  experiment  is  of  the  "two  factor,  one  response" 
type  (Model  III),  the  questions  of  interest  arc  similar  to  those  arising 
in  univariate  normal  analysis  of  variance,  i,.£.  ,  how  do  the  factors 
(cf .  treatments  or  independent  variables)  combine  to  determine  the 
response  (cf .  "yield"  or  dependent  variable.)?  Here  the  hypotheses  of 
interest  include  total  homogeneity  (_i.£. ,  do  both  factors  affect  the 
distribution  of  the  response?),  partial  homogeneity  (i.e. ,  does  one 
factor  affect  the  distribution  of  the  response?)  ,  and  "no  interaction" 
between  factors  in  the  way  they  affect  the  response  (i.e.,  do  the 
factors  determine  the  response  in  a  purely  additive  or  multiplicative 
way  or  are  the  relationships  between  the  factors  and  the  response  more 
complex?). 

For  completeness,  it  should  be  noted  that  for  Model  IV  situations 
("multi-factor,  multi-response"  experiments  and  hence  d  >  3) ,  interest 
centers  in  both  the  relationships  among  the  responses  and  in  the  way  the 
factors  combine  to  affect  the  responses.  Thus,  the  structure  of  the 
table  (which  is  not  always  obvious  or  unambiguous)  dictates  the  types 
of  questions  that  are  of  interest  in  a  given  experimental  situation.  In 
the  next  section,  these  concepts  will  be  formulated  in  terms  of  a  general 
mathematical  model  from  which  test  statistics  may  be  derived.  Tt  is  at 
this  point  that  various  differences  arise  among  the  three  general  method¬ 
ologies  based  on  minimum  XV  ,  maximum  likelihood,  and  minimum 
discrimination  information. 

2.2.  Hypotb.cs is  Formu ilat i on 

Let  tt.j  represent  die  probability  of  an  observation  from  the  i-th 
population  (or  factor  level)  falling  in  the  j-tli  response  category  (_i.e., 
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the  it  represent  individual  cell  probabilities).  IT  the  sampling 
is  either  from  a  very  large  population  or  with  replacement,  then  the 
probability  distribution  of  the  observed  frequencies,  n  ,  is  given 
by  the  product-multinomial  distribution.  This  model  lias  the  form 


r  s  (it  .  , ) 

4>  =  n  n,  !  n  tJ 

1-1  i0  j=i  rTTTT" 


(2.2.1.) 


where  £  IT  =  1  and  £  n..  =  n.  (fixed)  for  all  i  and  where  it  is 

j-1  ij  J-l  lj  10  - 

assumed  tnat  ir  >  0  for  all  i,  j.  This  basic  model  allows  various 
hypotheses  of  interest  to  be  expressed  as  functions  of  the  unknown 
parameters  in  (2.2.1).  Hence,  it  is  the  primary  basis  of  inference  in 
the  remainder  of  this  paper.  However,  it  should  be  noted  that,  if 
sampling  is  without  replacement  trom  a  relatively  small  population  and 
hence  the  sampling  fraction  is  large  (say,  exceeding  10  per  cent), 
methods  of  analysis  based  on  the  theory  of  sampling  from  finite  popula¬ 
tions  are  more  appropriate.  The  details  of  the  analysis  for  this 
situation  which  requires  adjustments  for  finite  population  correction 
factors  have  been  considered  by  Johnson  and  Koch  [1970b]  and  arc  illustrated 
in  Section  5.5. 

A  general  formulation  for  many  hypotheses  of  interest  in  the  analy¬ 
sis  of  categorical  data  (£.£.,  in  two-way  tables,  independence  in  the 
Model  I  situation,  homogeneity  in  the  case  of  Model  II)  is  given  by  the 
following  set  of  constraints  on  the  ceii  hr  ’'abilities: 


f  (it) 
t*l  ' 


(2.2.2) 


where 
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(f(TT))’  =  (f^TT),  f2(TT),  ....  ft(j))  t  <  r(s-l) 


(2.2.3) 


with 

j'  “  ,1T12  *  *  *  ‘  ,1Tls  ’  7T21,1T22’ ‘  ’  ,,T2s:  "rl  ,TTr2  ’  “  *  ,7Tvs^ 

=* 

Here,  the  functions,  f^(TT) ,  are  assumed  to  be  functionally  independent 

s 

of  each  other  and  independent  of  the  constraints,  E  tt  .  .  =  l ,  i  =  1,  2,. 

j=l  1J 

In  addition,  the  f^(ir)  must  have  continuous  first  and  second  partial 

derivatives  with  respect  to  the  tt.^s  ana  H  (u)  =  (3f  (tt)  /3tt  .  .  )^_ 

' J  txrs  "  "  1-)  ~  ~ 

must  be  of  rank  t  (i.e.,  full  rank)  for  any  u  in  the  neighborhood  of  it. 
For  example,  consider  the  hypothesis  of  independence  in  a  two-way 
contingency  table  of  the  "no  fac.or,  two  response"  type  (i_ . e_. ,  Model  I). 
This  hypothesis  is  usually  formulated  as 


•  ,r 


where 


and 


TT.  .  =  TT.  Tt  . 

J1J2  Jl°  °J2 


TT. 

Jl° 


represent  the  appropriate  marginal  probabilities.  Alternatively,  thi$ 
hypothesis  may  be  written  in  the  form  specified  by  (2.2.2)  as 

loge 

It  should  be  noted  that  since  this  situation  involves  a  single  multi¬ 
nomial  population,  the  i-subscript  has  been  suppressed.  Alternatively, 
if  the  two-way  contingency  table  is  of  the  "one  factor,  one  response” 
type  (i_.£. , Model  II),  then  the  hypothesis  of  homogeneity  across  sub¬ 
populations  may  be  written  in  the  form  specified  by  (2.2.2)  as 

tt  .  .  -  r.,.  =  0  for  all  i  ^  i'  and  each.  j. 

1  J 


.  .  TT.  ,  .  , 

J1J2  Jl32 
,  .  .  tt  .  ,  . 
J1J2  Jl:12. 


=  0 


3 1  T4  j  {  ,  j  ,  ^  j  2  • 
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In  other  cases,  the  f^Cif)  pertain  to  any  type  of  function  about  which 
the  researcher  desires  to  draw  inferences. 

More  generally,  hypotheses  involving  the  functions  f (it)  can  be 
formulated  in  terms  of  linear  regression  models 

H  :  f  (tt)  -  X  6  (2.2.5) 

t*l  ~  txu  uxl 

where  f(ir)  is  as  before,  X  is  a  known  design  matrix  of  rank  u  t  and  6 
is  a  vector  of  unknown  parameters.  Given  that  a  linear  model  for  the 
functions  f (tt)  applies,  hypotheses  concerning  various  constraints  on  the 
model  parameters  can  be  formulated  as  follows: 

HQ3:  C  3-0  (2.2.6) 

exu  uxl  cxl 

where  C  is  a  matrix  of  known  constants  of  rank  c  <_  u.  Proper  choices 
of  C  allow  hypotheses  pertaining  to  various  main  effects  and  lower  order 
interactions  to  be  examined.  This  aspect  of  analysis  is  illustrated  by 
several  examples  in  Section  5. 

2.3  Estimation  Procedures 

If  the  hypothesis  of  interest  is  formulated  in  terms  of  constraints 
on  the  cell  probabilities  as  in  (2.2.2),  then  it  is  necessary  to  obtain 
estimates  of  the  cell  probabilities  subject  to  those  constraints.  These 
constrained  estimates  are  then  incorporated  into  one  of  several  asymptoti¬ 
cally  equivalent  test  criteria  and  the  test  is  performed. 

In  the  framewotk  of  (2.2.2),  there  are  essentially  two  different 
classes  of  estimation  procedures.  These  are 

(1)  Maximum  likelihood  and  minimum  discrimination  information; 

o 

(2)  Minimum  chi-square  (minimum  y“)  and  modified  minimum 

2 

chi-square  (minimum  X|)  in  the  sense  of  Neyman. 
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Both  classes  of  estimation  procedures  satisfy  a  certain  optimality 
property;  namely,  they  all  yield  BAN  (best  asymptotically  normal) 


estimates.  Thus,  these  estimates,  e  ,  are 

(1)  Consistent  and  hence  asymptotically  unbiased  (!•£•,  they 


converge  in  probability  to  the  n^); 


io 


(ii)  Asymptotically  normal  as  N  «  I  n  ■*  "  with  —  ■=  £. 

i-1  N 

where  is  a  constant; 

(iii)  Efficient  (i.£. ,  the  variance  of  any  other  consistent, 

asymptotically  normal  estimate  is  at  least  as  large  as  the 


variance  of  n  ) ; 

*  J 


^TT 


ii 


(iv)  Sufficiently  regular  fi_.c. ,  ^  exist  and  are  continuous 

"  ij 

in  h  for  all  i , j ) . 

Historically,  maximum  likelihood  estimates  (MLE's)  were  probably  the 
first  estimates  derived  for  (2.?. 2)  and  have  been  fairiy  popular  in  certain 
special  contexts  like  quantal  response  bioassays.  To  obtain  Ml.tl's  of  the 
(2.2.1)  must  be  maximized  with  respect  to  tiie  rn^'s  and  subject  to 
(2.2.2).  This  leads  to  a  system  of  simultaneous  equations  which  are  often 
non-linear  in  the  and  consequently  difficult  to  solve.  Although 
various  iterative  schemes  have  been  proposed  (e.£.  ,  Roy  and  Kastenbaum, 
[1956!;  Kastenbaum  and  l.nmpii iear ,  (19591).  the  variety  of  possible  hypotheses 


has  precluded  the  formulation  of  a  general 
systems  of  equations.  Howevc-r,  due  to  the 
general  maximum  likelihood  niocedures  have 
Include  the  various  iterative  proportional 
in  Hosteller  [1968],  6.. oilman  [  1969,  1970], 
and  Hishop  [1969].  These  are  based  on  the 


solution  of  the  resulting 
results  of  Birch  [1963],  fairly 
recently  appeared.  These 
fitting  schemes  presented 
Bishop  and  lionborg  [1969], 
realization  that  for  certain 
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hypotheses,  the  MLE's  of  the  expected  cell  frequencies  are  uniquely 
determined  by  the  marginal  totals  being  equal  to  the  MLE's  of  their 
expectations. 

Alternatively,  Kullbnck  and  his  associates  have  recommended  the 
use  of  minimum  discrimination  information  estimates.  These  BAN  estimates 
arc  obtained  by  minimizing,  with  respect  to  the 


r  s 


I(ntp*)  -  I  £  in(-~J) 

«•  •*  ....  1  1  D  .  . 


i-l  j-i  pij 


(2.3.1) 


subject  to  (2.2.2)  where  the  are  fixed  by  hypothesis,  observed, 

or  estimated.  Similar  to  maximum  likelihood  estimation,  this  procedure 

also  usually  requires  iterative  procedures. 

2 

Minimum  x  estimates  are  obtained  by  minimizing 

.  2 


;  (nn  ~  vu* 


i*i  j*i 


nio"ij 


(2.3.2) 


with  respect  to  the  n^'s  and  subject  to  (2.2.2).  Again,  this  technique 

usually  involves  the  solution  of  a  system  of  complicated,  non-linear 

2 

simultaneous  equations,  and,  since  minimum  x  estimates  are  not  known 
to  possess  any  desirable  properties  that  MLE's  lack,  these  estimates  have 
seldomly  been  used. 

If  the  denominator  of  (2.3.2)  is  replaced  by  the  observed  cell 
frequency,  n^,  a°d  the  resulting  sun 


5  !  (nn  ~  nioniir 

i-i  j-i  nij 

minimized  with  respect  to  the  tt  ^  ^  *  s  and  subject  to  (2.2.2),  the 
2 

resulting  minimum  estimates  are  obtained  by  solving  only  linear 


(2.3.  1) 
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equations  provided  the  fk(tr)  at*  linear  in  the  ^ ' s -  Neyman  (1949] 
proved  that,  if  the  f ^ (it)  are  not  linear  in  the  j  ’ s ,  the  f^u)  may 
be  replaced  by  their  Taylor  series  expansion  about  the  point,  ir  ■  p, 
where  p^  -  l<  1 .  ri  J  1  1 . .  namely 


“  ffc(p) 


r  s 
+  Z  Z 
i-1.  j-1 


3fk(n> 

(  - 


3  it 


ij 


Tr-P 


(it 


ij 


'  PU) 


(2.3.4) 

k  -  1,  2,  .  .  .  ,  t 

thereby  reducing  the  problem  to  the  linear  case.  In  either  case,  the 
2 

resulting  minimum  estimates  are  BAN  estimates  and  do  no t  require 
Iterative  procedures'  for  their  solution. 

In  the  framework  of  (2.2.5)  where  a  linear  model  is  fitted  to 
the  data,  Gri22le  ct  al.  [1969]  estimate  £ (tt )  by  replacing  i»  by  its 
unrestricted  MLE,  p,  and  then  estimate  the  parameters  (g)  of  the  model 
by  ordinary  weighted  least  squares  procedures  applied  to 


f  (p)  -  X  g  .  (2.3.5) 

t*l  ~  t<u  u*l 

This  procedure  yields  the  familiar  weighted  least  squares  estimate 
(tP-St),  b,  of  g,  namely 

b  -  (X’S'Sc)'1  x’S"1f(p)  (2.3.6) 

u*  L  ~  *  “  '  '  ~  ' 

where 


S  a  a  consistent  estimate  of  the  covariance  matrix  for  f(p) 
txt  '  ' 

Th.s  estimation  procedure  Is  discussed  in  r.ore  detail  in  Section  3. 
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2.4  Test  Statistics 


A  number  of  asymptotically  equivalent  test  statistics  have  been 
proposed  for  testing  the  hypotheses  specified  by  (2.2.2)  and  (2.2.5). 
Asymptotic  equivalence  requires  thtit ,  regardless  of  whether  the  null 
hypothesis  holds,  the  probability  of  any  two  tests  being  contradictory 

r  n 

tends  to  rero  as  N  -  £  n.  ■*  ”  with  — rr-  -  £ ,  where  £.  is  a  constant. 
i-1  io  Nil 

The  test  statistics  for  (2.2.2)  fall  into  three  essentially  different 
classes.  They  are 

(1)  Pearson's  chi-square  statistic 


(2)  (a) 


(b) 

6)  (a) 


(b) 


,  r  s  (n  -  n  tt  ) 
y2  m  r  v  _ *1  to  11 

A_  w  w  -■v 

i-1  1-1  n  tt 

J  io  ij 


(2.4.1) 


Ncyman-Pearson 's  likelihood  ratio  chi-square  statistic 
r  s 

z  z 

u  i-1  J-l 

Minimum  discrimination  information  statistic 


i 

x;  -  I  I  (  -  2  In  (  1  j  I  . 


(2.4.2) 


r  s 


•  12N  v?  £n(Ji)). 


i-1  j-l 


ij 


ij 


Neyman's  chi-square  statistic 


x:  -  T.  T. 

N  i-i  j-l 


nu 


(2.4.3) 


(2.4.4) 


Wald's  statistic 


-  (t  (p)  )'  s'1  <f  (P)  ) 


(2.4 .5) 
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wh«r«  p,  £(p),  S,pt  and  N  are  defined  in  Section  2.3,  the  are 

fy 

/\ 

any  BAN  eatlmates  of  the  n^,  and  the  it  are  the  minimum  discrimina¬ 
tion  information  estimates  obtained  by  minimizing  (2.3.1). 

For  testing  the  fit  of  the  linear  model  (2.2.5),  the  usual 
analysis  of  variance  error  sum  of  squares  is  used,  i,.e. , 

xj  -  ss(f  (n)  -  X6)  -  (f  (p) ) '  s-1(f(r))  -  (xh)'  s'1  (Xb) 

(2.4.6) 

where  b  is  defined  in  Section  2.3.  Then,  given  that  the  model 

adequately  fits  the  data,  tests  involving  contrasts  of  the  model 

parameters  (  £  £  ■  0  )  are  produced  by  usual  analysis  of  variance 

c  xu  u  xl  c  xl 
sums  of  squares,  j^.e.  , 


SS(C8  -  0) 


(Cb)  ’  (C(:<,S_lX)~1t:'  ]_i(Cb) 


(2.4.7) 


whore  C  is  a  matrix  of  constants  of  rank  c  <  u. 


->  -1 


Under  in  (2.2.2),  Ney-an  [1949]  has  shown  that  X",  and 

2  2 
X^,  using  any  BAN  estimates  of  the  n^,,  are  asymptotically  >.  variates 

with  t  degrees  of  freedom  (D.F.)  and  hence  are  asymptotically  equivalent 

provided  that 
nto 

(l)  i  •  1,  2,  r,  remain  constant  as  N  ■  •■= , 

1% 

(ii)  f^Om)  "  0,  k  «  1,  2,  ....  t,  has  at  least  one  solution  such 
that  >  0  for  all  1,  j. 


Bhapkar  [1966]  has  shown  that  X.*-,  (Wald's  statistic)  .'  .  testing  linear 

Vs 

> 


hypotheses  in  categorical  data  is 
2 

Xj^  is  defined,  i,.e.  ,  whenever  all 


algebraical 


the  n^ave 


lv  identical  to  X"  vhenevi.r 
positive;.  Similarly,  for 
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2  2 

testing  non-linear  hypotheses,  Xy  Is  identical  to  using  Ncyman's 

linearization  technique  on  the  hypothesis  constraints  as  In  (2. 3. A). 

2 

Kullback  [1959]  has  shown  that  (the  minimum  discrimination  information 

2 

statistic),  under  is  also  asymptotically  y  with  D.F.  ■  t  so  that 

all  of  these  testa  used  for  are  asymptotically  equivalent  when  using 
the  appropriate  estimates  of  the  individual  cell  probabilities. 

In  the  context  of  '.he  linear  model  as  in  (2.2.5),  the  tests  are 
derived  by  conventional  methods  of  weighted  least  squares  and  hence  are 

the  sane  as  Neyman's  chi-square  tests  when  translated  into  constraints. 

2  "> 

Under  H^,  for  testing  the  fit  of  the  model  is  asymptotically  y“ 

2 

with  D.F.  ■  (t  -  u) ;  given  the  model,  tor  testing  contrasts  of  the 

2 

model  parameters  is  asymptotically  with  D.F.  *  c  under  the  null 
hypothesis,  C6  •  0. 

3.  THE  LINEAR  MODF.L  APPROACH 

3 . 1  Notation  and  Assumption 

In  this  section,  the  general  methodology  for  categorical  data 
discussed  by  Grizzle  ec^.  al.[1969J  will  be  outlined  in  more  detail.  The 
underlying  probability  model  is  that  defined  by  (2.2.1)  with  the  notation 
referring  to  the  expected  cell  probabilities  and  hypothetical  data  shown  in 


Table  3.1.1. 


Table  3.1.1 


Expected  cell  probabilities  (cell  frequencies)  for  the 
standard  contingency  table 


I - - - - -  — 1 

Populations 

Response  Categories 

Totals 

(factors) 

1 

2 

8 

1 

V 

TT 

it 

1 

11 

12 

is  ! 

I 

I 

1 

("11 ) 

1 

(ni2) 

<nls) 

(nlo} 

2 

1 

TT 

T*  1 

1  1 

(  21  * 

22 

Is 

1 

(n21 

(n22) 

(n2  )  ; 

i 

<"2o> 

•  1 

.  1 

r 

• 

j  *rl 

%2 

\ 

...  TT 

rs 

1 

_ i 

(nrl> 

i _ 

(nf2) 

("rs> 

(n  ) 
ro 

Let  n '  be  defined  as  in  (2.2.4)  and  let  p'  be  the  corresponding  vector 
of  unrestricted  MLE's  of  the  respective  components  of  t';  i^.e.  , 


E'"^pll  ,p12  *  ”  ‘  ,pls  :p21’P22’'  ’  ’  ,P2S  ’  ;f>rl  ,Pr2”  ‘  *  ,Prs) 
lyrs 

-<P1,.P2‘ . pr  ' } 

where  ■  (n^j/n^o)  for  1  ■  1,  2,  ....  r  and  j  =  1,  2,  ...,s. 


(3.1.1) 


Let  V  (^)  be  the  variance-covariance  matrix  of  the  unrestricted  ,'il.K's 


from  the  1-th  population;  hence, 
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J<V  -  v.r(E  >  -  — 

8*S  iO 


-*11*12  *“  -”11*1. 


*i2(1'*i2)  -*12*18 


(symmetric) 


*iS<l-*l9) 


0.1.2) 


A  consistent  estimator  for  V  (tt^)  independent  of  any  hypothesis  on  the  Tt ^ 


Is 


Yi  "  Y(pt> 

8X8 


V(n  )  •  sample  estimate  of  V(it  )  (3.1.3) 

'  -1  !i  ‘  Si  '  -1 


Hence, 


a  consistent  estimator  for  the  variance-covariance  matrix  of  p  is 


V 

rsxrs 


V(p) 


block  diagonal  matrix  with  diagonal  blocks 


of  the  f err. 


^  1  \ 
-  i'  r 


(3.1.4) 


where 
Let  f 


-  diagonal  (pu.  pi2 . Pis>. 

be  defined  at;  in  (2.2.3)  and  define  H  and  S  by 


,fk<:} 

H  -  H(p)  -  (-p - )  (3.1.3) 

txrs  '  ii  nr-p 

S  ■  HVH1  •  sample  estimate  of  the  variance-covariance  matrix 
txt  of  f  (o)  after  linearization  as  in  (2.3.4) 

(3.1.6) 


It  is  assumed  that  the  f  (r )  are 


and  of  the  constraints,  Z  m  “ 

j-1  lJ 


functionally  independent  of  each  other 
1,  for  i*l,2,.  .  .,  r;  hence  H  and  HVH’ 


are  asymptet leal lv  of  full  rank.  Finally,  if  certain  »  0,  the  authors 
suggest  that  they  be  replaced,  in  certain  cases,  by  (1/s)  to  eliminate 
possible  singularities  in  S.  (See  Berkson  [1^53]). 
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3.2.  Methods  of  Inference 

Assume  that  f(it)  ■  X3  where  the  f^’s  are  possibly  non-linear  and 
X  end  $  ere  defined  following  (2.2.5).  For  the  no  factor,  multi -response 
case  (i.e. ,  one  population  problems),  the  relevant  hypotheses  are  those 
of  various  types  of  independence  and  symmetry.  These  are  tested  using 

Xp  >-■  SS(f (it)  -  0)  -  f’s"1  f  -  xj  (3.2.1) 

2 

which,  under  f(n)  •  0,  is  asymptotically  x  with  D.F.  ■  t.  If 
f (tt )  is  linear,  i,.£. ,  f (tt )  ■  An  where 


a(i)ll  ’•*  a(l)ls;  a(l)2r”  a(l)2s’ 

“(2U1  a(2)ls:  a(2)2r"  a(2)?s: 

A  ■ 

t~*T8  *  ... 

•  •  •  • 

.  .  •  • 

_°(t)ll  a(t )  Is  ;  a(t)21’  au)2s; 


a(l)rl‘  a(l )rs 
a(2)rl  aC)rs 

a(t  )r  1 '  ‘  '  a(t)rs_ 


ie  an  appropriate  matrix  of  constants  of  rank  t,  then  (3.2.1)  is  nore 
explicitly  given  by 
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A  second  class  of  functions  often  arising  in  categorical  data  problems 
is  the  class  consisting  of  the  logarithmic  functions  which  can  he 
expressed  by  f(r)  *  K  f.n(A7r)  where  .e,n(Vir)  denotes  the  vector  of  natural 

(or  Napcrian)  logarithms  of  the  elements  of  An ,  and  II  =  (k  )  is  an 

vxt  cc/ 

appropriate  matrix  of  constants  of  rank  v  <  t.  The  corresponding  test 
statistic  for  this  case  is  given  by 

Xjl  =  SS  ( f  (tt  )  =  K  fn  (An )  =  0)  =  (3.2.3) 

(K  q,n(Ap))'  (KD_lAVA’ir1K')~1  (K  f.n(Ap)) 

where  p  =  diagonal  (a'^^.p,  a’^.p,  ...,  a'^.p).  Under  liJj,  X"  is 
2 

asymptotically  y"~  with  P.F.  =  v.  It  should  he  noted  that  caution  must 
be  taken  in  constructing  A  so  that  no  element  of  the  vector.  \p.  is 
zero.  Often  when  logarithmic  functions  are  used,  there  is  interns'  in 
multiplicative  models  for  ”  and  hence  A=T= ( ident i ty  matrix).  Since,  empty 
cells  are  to  he  replaced  by  L/s.  each  element  of  Ip  (=  p)  will  be  positive 
so  that  no  difficulty  is  encountered  with  tin's  common  application  of  the 
logarithmic  functions. 

For  example < cons ider  the  Lost  of  independence  for  the  two-way  case 
cited  in  section  2.2.  Let  =  !.  2:  '>.,  =  1,2.  Then 

ilQ:  f(r)  *  -(.V)  =  0 


is  given  by 


f(v)  =  n  <  ' 
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so  that 


For  the  uni-  or  multi-f actot «  uni-  or  multi-response  case  (l.e. 


several  population  problems),  X  is  known  and  of  rank  u  ^  t.  A  test 
of  fit  of  the  model  fC*)  ■  XB,  is  given  by  the  residual  sum  of  squares 

Xp  -  SS(f<n)  -  XB)  -  f'S_1f  -  (Xb)'s"'(Xb)  (3.2.5) 

where  b  is  the  vector  that  minimizes  (f  -  Xb)'S  1 < f  -  Xb)  and  is  given 
by 

b  «  (X,s'1X)"'LX’S'1f .  (3.2.6) 

?  2 

If  the  model  fits  the  data,  is  asymptotically  X  with  D.F.  •  t  -  u. 
Recalling  that  for  the  linear  case,  f  «  Ap,  S  *  AVA*  and  for  the 
logarithmic  case,  f  *  K  in(Ap),  S  *  KD  ^AVA'D  *K',  explicit  expressions 
tor  the  test  statistic  given  in  (3.2.5)  can  easily  be  obtained  for 
these  two  classes  of  functions. 

Given  that  the  model  fits,  various  hypotheses  concerning 
constraints  on  th^  model  parameters,  ,  ....»>•  be  of  interest.  The  choice 
of  hypotheses  is  illustrated  in  considerable  detail  in  section  5.  A  test 
of  the  hypothesis,  Kq3;  Cfi  ■  0 ,  is  produced  by 

x£  -  SS(C6  -  0)  -  (CbVtCCX'fScrVf'fCb)  (3.2.7) 

where  C  is  a  (exu)  matrix  of  arbitrary  constants  of  rank  c  ^  u.  under 
2  2 

HG3’  *C  a,XolPtotically  x  with  D.F.  •  c.  Again,  explicit  expressions 
for  (3.2.7)  can  easily  be  obtained  for  the  linear  and  logarithmic  classes 
of  functions. 
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4.  MORE  GENERAL  CATEGORICAL  DATA  MODELS 


(11)  The  data  is  complete  In  the  sense  that  everv  experimental 
unit  Is  classified  according  to  each  of  tin  J  dimensions 
of  the  table. 


In  many  practical  problems,  some  of  these  assumptions  must  be  relaxed. 
Goodman  [1968]  and  Bishop  and  Fienberg  [1969],  among  others,  have 
considered  (i)  by  dealing  with  the  implications  of  having  a  priori 
certain  empty  cells.  Their  approach  utilises  multiplicative  models  with 
maximum  likelihood  estimation.  The  relaxation  of  (i)  for  the  linear  model 
so1'  "na  o’  -  if  handle-!  vv  suitable  definition  of  i  (it)  and  is  discussed  in 
CrizzJe  -ind  Williams  (1970].  The  relaxation  of  (ii)  is  the  primary  con¬ 
cern  of  this  section. 


In  the  usual  categorical  data  situation,  there  is  complete  infor¬ 
mation  available  for  each  entry  in  the  corresponding  contingency  table. 

For  instance,  in  a  four-dimensional  tanle  ot  t lie  two-lactor,  two-response 
type,  each  experimental  unit  has  been  classified  according  to  all  four 
dimensions.  However,  in  many  situations,  the  experimenter  is  not  equally 
interested  in  the  response  variates  or  perhaps  additional  information  on 
one  particular  dimension  could  be  obtained  relatively  easily  and  economically. 


Alternatively,  perhaps,  as  described  in  Kleinbaum  [1970]  for  the  con¬ 
tinuous  case,  there  is  missing  data  for  some  individuals  In  the  sense 
that  classifications  on  only  d*  <  d  dimensions  have  been  made.  Retaining 
and  utilizing  this  partial  Information  is  distinctly  different  from  the 
"apparently"  similar  problem  of  empty  cells  (see  Assumption(i)) . 

Here,  the  additional  information  is  assumed  to  arise  by  design 
rather  than  by  chance  and  as  such  is  referred  to  as  "supplemental 
information"  and  the  corresponding  margins  of  tlr  contingency  table  as 
"supplemented  margins".  This  supplemental  information  is  used  to 
improve  the  precision  of  tue  estimates  of  the  marginal  probabilities 
over  the  supplemented  dimension(s) ,  and  it  is  also  incorporated  into 
the  estimation  of  the  individual  cell  probabilities  in  order  to 
Improve  certain  tests  of  hypotheses. 

Kxamples  where  such  supplemental  information  might  arise  include 
questionnaire  surveys  similar  to  the  recently-completed  national 
census.  In  such  surveys,  every  respondent  would  b.  expected  to  answer 
a  basic  set  of  questions.  In  addition,  a  certain  proportion  would  be 
asked  to  respond  to  a  second  set  of  questions  judged  to  be  less  impor¬ 
tant  or  more  difficult  to  arswer  than  the  questions  in  the  basic  set. 
Supplemental  information  would  derive  from  the  answers  to  questions  in 
the  basic  set  from  those  respondents  not  icquested  to  consider  the 
second  set  of  questions.  Similarly,  1 r  automobile  crash  investigations, 
the  investigating  officer  might  routinely  obtain  certain  biographical 
information  (s.£.  ,  race,  sex,  age)  for  all  drivers  tut,  due  to  the 
difficult  conditions  of  the  interview,  obtain,  onl\  for  a  specified 
subset,  additional  information  such  as  sobriety  of  the  driver,  purpose 


of  Che  trip,  etc.  In  this  case,  the  supplemental  Information  would 
consist  of  the  answers  to  those  biographical  questions  for  those 
drivers  who  were  not  examined  In  detail. 

It  should  be  noted  here  that  the  use  of  supplemented  margins 
In  the  analysis  of  contingency  tables  resembles  the  technique  of  "double 
sampling"  as  described  In  Cochran  (1963).  In  "double  sampling  ">a 
preliminary  sample  (cf .  supplemental  Information)  is  used  to  estimate 
the  distribution  of  an  auxiliary  variable  ( cf_ .  marginal  distribution 
of  particular  response  or  combination  cf  responses)  and  a  second 
stratified  sample  is  used  to  estimate  the  characteristic  of  interest 
(cf .  cell  probability!. 

Some  work  In  this  area  has  recently  been  done  by  Blumenthal 
[1968]  who  treats  essentially  the  same  problem  as  described  here. 
However,  he  restricts  attention  to  one-way  tables  where  each  of  the  I 
main  categories  has  Ji>  1  ■  1,  2,  ...,  I,  sub-categories  of  classifi¬ 
cation.  In  addition,  Blumenthal  assumes  that  a  sample  unit  belonging 
In  cell  (i,j)  has  probability  Ct^  of  being  only  "partially  classified" 
as  opposed  to  having  a  secondary  sample  for  which  supplementary  infor¬ 
mation  on  a  subset  of  the  study  variables  is  obtained.  Imposing  certain 
simplifying  assumptions  for  this  one-way  situation,  Blumenthal  obtains 
MLE's  of  the  individual  cell  probabilities  as  well  as  the  bias  and 
variance  of  these  estimates. 

It  should  be  noted  that  supplementation  is  considered  only  for 
Margins  that  are  nor  fixed  (_i.e.  ,  response  margins  rather  than  factor 
margins).  Supplementing  a  factor  margin  is  of  no  interest  since  that 
margin  being  considered  fixed  a  priori  yields  complete  information 
about  its  composition. 
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In  order  to  ae*  more  clearly  the  implication*  of  ualng  supple- 
mental  data,  let  u*  consider  the  special  case  corresponding  to  the 
simple  2x2  contingency  table  of  the  "no  factor,  two  response"  type. 

Let  A  oanote  one  of  the  response  variables  and  let  B  denote  the  other. 
If  subjects  are  classified  according  to  both  A  and  3  while  n  „  are 
classified  only  according  to  A  and  n^  are  classified  only  according  to 


B,  the  resulting  data  can  be  displayed  aa  in  Table  4.1.1.  It  is  assumed 

that  there  is 

Table  4.1.1. 
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no  interaction  between  subjects  and 
i^£. ,  for  the  responses  classified, 
lubjects  with  complete  data  and  for 
same.  Conditional  on  the  number  of 


the  presence  of  supplemental  data; 
the  joint  marginal  distributions  for 
those  with  supplemental  data  are  the 
individuals  with  various  types  of 
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compl«te  or  supplemental  data  being  fixed  by  design  a  priori  (or  by 
assumption  a  posteriori) .  the  basic  probability  model  is  the  product 
of  several  multinomials  line  (2.2.1)  with  marginal  and/or  cell  proba¬ 
bilities  as  fundamental  parameters.  For  the  example  in  Table  A. 1.1, 
this  model  Implies  that 
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(A. 1.1) 


and  that  a  consistent  estimate  for  the  variance  of  the  various 
frequencies  is  V  as  given  by  -j 

nooPll(1_Pll)  -nooPUP12  -nooPllP21  °  0 

°ooP12 (1_P12)  -noop12P21  0  0 

nooP21(1-p21)  0  0 

symmetric  n0*pl*(1"pl*)  0 

(A. 1.2) 
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where 


P1  1  '  (ni  i  /n 00 pi  =  (nl  /no»'<)'  P*1  "  <n*i 
J1.J2  J.1J2  -.1*  J2  J2 


j 1 ,  ■  1*  2.  Rcinfurt  [1970]  indicates  how  the  supplemental  infor¬ 

mation  can  be  used  to  obtain  improved  estimators  for  the  parameters 
wu,  ^22’  ir-)i»  He  discusses  both  maximum  likelihood  estimation 

and  weighted  least  squares  estimation  (which  if  iterated  leads  to  esti¬ 
mators  which  are  equivalent  to  the  MLE's).  Either  of  these  estimators 
for  the  71  j ^ j 2  can  used  to  play  the  same  role  ns  the  unrestricted  Midi's 
in  t  lie  unsupplcmented  case.  This  means  that  one  can  test  hypotheses  of 
the  'orm  (2.2.2)  or  (2.2.5)  by  using  the  linear  model  approach  of  Section  3. 
All  that  is  required  is  to  formulate  appropriate  definitions  of  the  A, 

A 

K,  X,  and  C  matrices  and  to  replace  V  as  in  (3.1.4)  by  the  consistent 

estimator  of  the  asymptotic  covariance  matrix  for  the  estimators  of 

the  11 .  .  which  have  been  adjusted  for  supplementation.  All.  of  this 
J1J2 

can  be  put  together  in  terms  of  a  two-stage  test  procedure  utilizing 
the  computer  program  described  by  l-orthofer  et :.nl .  f  19(>9]  as  follows. 

The  first  stage  consists  of  the  following: 

1.1  Input  the  observed  cell  and  marginal  frequencies. 

1.2  Obtain  estimates  of  the  cell  probabilities  corresponding 
to  the  unrestricted  MLr's  for  the  case  with  complete  data: 

(a)  Tf  the  table  Ls  fairly  simple,  obtain  the  Midi's. 

(This  is  the  closest  analogue  to  the  unrestricted  Midi’s 
for  the  complete  data  version  and  is  most  consistent 
with  tests  using  Wald  statistics.) 

(b)  lor  most  cases,  compute  the  weighted  least  squares 

estimators  by  Iterative  techniques. 
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(c)  If  Interested  In  estimating  functions  of  the  cell 
probabilities  (sey,  An),  rsther  then  of  tt  obteln 
weighted  leest  squares  estimators  of  these  express¬ 
ions  directly.  This  Is  particularly  appropriate  when 
sample  slaea  are  small. 

1.3  Obtain  an  estimate,  0  'or  A$A’  for  (c)>,  of  the  asymptotic 
covariance  natrlx  using  the  estimates  derived  In  (1.2). 

The  second  stage  consists  of  the  following: 

11. 1  Input  the  A.  K,  X,  and  C  matrices  required  for  the  analysis 
as  In  tt\e  existing  program. 

11. 2  Use  the  adjusted  (for  supplemental  data)  estimates  of  the 
cell  probabilities  as  obtained  in  (1.2)  of  the  first  stage 
in  place  of  the  usual  estimates  along  with  the  adjusted 
estimate  of  the  asymptotic  covariance  matrix  in  (1.3)  of 
the  first  stage  In  place  of  the  usual  estimate.  Proceed 
with  the  testing  of  various  hypotheses  exactly  as  In  the 
existing  linear  models  framework. 

An  example  of  this  two-stage  procedure  is  given  in  Section  5.4. 

4.2  Contingency  Tables  from  Sample  surveys  of  Finite  Populations 
The  general  methods  of  analysis  discussed  by  Gristle 
et  al .  [19f>9|,  Goodman  {1970],  ar.d  Ku  and  Kullback  [1968]  presume 

that  the  underlying  model  for  complex  contingency  tables  Is  the  pro¬ 
duct  multinomial  distribution.  This  model  does  not  directly  apply  to 
categorical  data  arising  from  sample  survey  situations  sinre  the  sel¬ 
ected  subjects  give  responses  which  can  be  often  either  determined  with 
complete  certainty  (e.£.  ,  sex,  amount  of  education,  year  of  birth,  years 
of  experience)  or  which  have  widely  varying  probability  distributions 
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(«.&.,  opinions  regarding  suitability  of  csrtaln  typas  of  health 
ssrvlces  or  programs ,  the  occurrences  of  accidents  or  diseases 
during  specified  time  periods,  the  occurrence  of  a  positive  or  other 
type  of  response  to  various  methods  of  therapy).  In  addition,  if 
the  actual  survey  design  Is  a  complex  highly  stratified  cluster  scheme 
with  varying  probabilities  of  selection,  the  underlying  probability 
distribution  becomes  considerably  more  difficult  to  handle. 

In  spite  of  these  problems,  it  does  follow  from  the  theory  of 
sample  survey  methodology  that,  in  large  samples,  the  estimated  pro¬ 
portions  in  coriespording  complex  contingency  tables  have  approximately 
multivariate  normal  distributions  with  a  covariance  matrix  which  can  be 
consistently  estimated  by  straightforward  nnd  well-known  methods  (as 
developed  in  Cochran  (1963]).  Since  this  fact  is  really  the  crucial 
assumption  underlying  the  use  of  the  approach  of  Grizzle  JLL  *L* 

11969]  in  an  ultimate  sense,  all  that  la  required  to  apply  their 
results  to  categorical  data  from  sample  surveys  is  to  make  certain 
appropriate  modifications  consistent  with  the  principles  of  survey 
sampling. 

Some  progress  has  been  made  in  this  direction  for  the  case  of 
stratified  simple  rsr.dom  samples.  A  number  of  results  based  on  the 
linear  models  approach  have  been  described  in  Johnson  and  Koch  (1970b) 
and  are  illustrated  in  terms  ot  health  services  evaluation  data  obtained 
from  a  sample  survey  of  hospital  administrators.  Alternatively, 

Nathan  (1969,  1970]  has  proceeded  from  a  somewhat  different  point  of 
view  and  concentrated  substantial  effort  on  tests  for  the  hypothesis 
of  independence  in  two  way  contingency  tables  derived  from  stratified 
samples . 
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Finally,  it  should  be  noted  that  for  this  situation  the 
analysis  may  be  viewed  in  a  two-stage  fashion  with  essentially  the 
same  structure  as  that  described  in  Section  4.1.  However,  here  the 
estimates  of  the  parameters  at  the  first-stage  are  developed  according 
to  the  principles  of  sample  survey  methodology.  For  simple  survey 
designs,  such  estimates  are  obtained  in  a  straightforward  fashion; 
in  more  complex  situations  with  unequal  selection  probabilities,  their 
derivation  as  well  as  the  corresponding  estimated  variance-covariance 

4 

matrix  is  more  difficult;  however,  once  the  first  stage  is  completed, 
the  second  stage  is  essentially  the  same  as  outlined  previously  and 
involves  only  the  proper  specification  of  A,  K,  X  and  C  matrices. 
Finally,  an  example  of  a  sample  survey  situation  is  described  in 
Section  5.5. 

5.  EXAMPLES 

5.1.  A  simple  quantal  biological  assay. 

One  type  of  experimental  situation  in  which  categorical  data 
often  arise  is  the  biological  assay.  The  purpose  of  such  investigations 
is  to  measure  the  potency  of  some  given  treatment  relative  to  some 
standard  treatment  in  terms  of  the  magnitudes  of  their  effects  on 
responses  from  living  material  (see  Finney  [1964]).  If  the  measured 
response  is  expressed  in  all  or  nothing  fashion  like  lives  or  dies, 
then  the  experiment  is  called  a  quantal  bioassay. 

An  hypothetical  example  of  the  results  of  a  quantal  bioassav 
is  given  in  columns  1-4  of  Table  5.1.1.  The  experimental  design  hero 
involves  assigning  20  animals  to  receive  each  of  the  specified  doses 
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of  Che  two  preparations  which  have  been  labelled  here  as  "standard" 
and  "treatment".  It  can  be  noted  that  as  the  dose  of  either  preparation 
is  increased,  the  proportion  of  animals  which  die  also  increases.  Ti¬ 
ls  this  relationship  between  the  proportion  of  dead  animals  and  dore 
for  which  a  categorical  data  model  is  needed.  One  model  which  has 
received  extensive  attention  in  the  literature  is  based  on  the  logistic 
function.  In  this  case,  it  is  assumed  that  as  dose  is  Increased  on 
the  log  dose  scale,  the  proportion  of  dead  animals  increases  according 
to  the  cumulative  probability  function  of  the  logistic  distribution; 
_i.£^,  if  P(x)  denotes  the  proportion  of  dead  animals  as  a  function  of 
log  dose  x,  then  the  assumed  model  is 

P(x)  -  (1  +  expf-Ca  ♦  3x) ) }_1 .  (5.1.1) 

A  second  assumption  which  is  made  is  that  the  treatment  preparation 

acts  like  a  dilution  of  the  standard  in  the  sense  that  there  exists  a 

parameter  p  called  the  relative  potency  of  the  treatment  such  that  for 

every  dose  u^.  of  treatment,  the  effect  on  the  proportion  of  d« ad  anirals 

is  the  same  as  the  dose  u  ••  ,  u_  of  standard.  In  terms  of  the  model 

s  I 

(5.1.1)  with  subscripts  T  and  S  referring  to  treatment  and  standard, 
this  means 

PT(xT  “  1°e10':T)  ’  PS(xS  “  (5.1.2) 

•  { l  +  exp( -(tg  +  3Xg) 1 )  1 

•  (1  ♦  expf-(as  +  3lo810C  ♦  .-Xj)}”1 

•  U  +  expl-foLj.  *  Bxt|:"1. 
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In  other  words,  the  functions  Pg  and  have  different  parameters  Og 
and  ctp  but  the  same  3.  Moreover  the  relative  potency  P  is  related  to 
the  parameters  as,  o^.,  6  in  the  sense  that 


log10P  -  (oT  -  «s)/  £ 


(5.1.3) 


It  should  ba  noted  here  that  log  dose  x  »  log  u  and  log  p  can  be 
expressed  in  terms  of  any  base  which  is  convenient.  Here,  iog^Q  has 
been  used  for  simplicity  since  log.j^(2)  '  0.3. 

The  model  implied  by  (5.1.1)  or  (5.1.2)  can  be  fitted  to  the 
data  in  a  very  simple  manner  as  soon  as  it  Is  recognized  chat 


1)  « 


-inf  exp[  -  (u  +  :•  x)  )  } 


(5.1.4) 


-  Cl  +  £  x 


P  (  X  ) 

where,  as  before,  in=loge-  The  quantity  Is  often  referred  to  as 

the  ''logit."  For  the  given  example,  its  values  appea?  in  column  6  of 
Table  5.1.1. 

The  validity  of  the  various  assumptions  can  be  tested  by  fitting 
the  following  linear  model  to  the  ve-r^r  of  logits 
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(5.1.5) 


A  statistical  test  for  the  goodness  of  fit  of  the  logistic  model 

(5.1.1)  is  provided  by  the  weighted  residual  sum  of  squares  which 
2 

in  this  example  is  X  «  0.75  with  D.F.  -  5;  hence  it  is  reasonable  to 
conclude  that  the  logiatic  model  is  appropriate  for  these  data.  The 
assumption  that  the  treatment  acts  like  a  dilution  of  the  standard  is 
equivalent  to  the  hypothesis  that  Pg  and  have  the  same  8  or  equiva¬ 
lently  that  the  lines  fitted  to  the  two  Bets  of  logits  in  (5.1.5)  are 

parallel.  This  can  be  formulated  here  in  terms  of  the  C-matrix  [001  -1] 

2 

as  described  in  Section  3.  The  resulting  X  ■  0.76  with  D.F.  ■  1  and 
hence  Justifies  the  dilution  assumption. 

Since  the  previously  described  tests  support  the  validity  of 
the  model  specified  by  (5.1.1)  and  (5.1.2)  together,  tie  final  step  is 
to  fie  the  reduced  model  with 


*2 


1  0  0.0 
1  o  0.3  ! 
1  0  0.6 
i  o  c.9  ; 
l  0  1.2  I 
0  1  0.0 
0  1  0.3 

0  1  0.6 

0  1  0.9 


(5.1.6) 


to  the  vector  of  logits.  The  goodness  of  fit  test  for  this  model  has 
2 

X  •  0.85  with  D.F.  ■»  6  which  further  verifies  our  previous  conclusions 
with  respect  to  Xj.  Estimates  for  the  parameters  in  (5.1.6)  are 

;s  -  -  2.60 

aT  •  -1.53  (5.1.7, 

S  »  4.34 


4?0 


and  predicted  values  for  the  logits  based  on  these  estimates  are 
given  in  column  8  of  Table  5.1.1;  the  corresponding  residuals  appear 
in  column  9.  Frou  the  estimates  in  (5.1.7),  it  follows  that  an 

estimate  for  log^p  19 

log^O  -  (clt  -  os)/S  -  (1.02)/  (4.34)  -  0.235. 

Finally,  by  using  the  C-matrices  [1  -1  0]  and  [0  0  1],  test 

statistics  for  the  hypothesis  of  equality  of  preparation  effects  and 

2 

equality  of  dose  effects  can  be  tested;  the  results  here  are  X  »  6.49 
2 

with  D.F.  «  1  and  X  ■  46.62  with  D.F.  •  1  respectively.  For  a  further 
discussion  of  quantal  bioassays,  the  reader  is  referred  to  Berkson 
[19351  and  Finney  [1964]. 

5 . 2  A  factorial  design  type  contingency  table  with  interaction. 

Many  higher-dimensional  contingency  tables  arise-  from  experimen¬ 
tal  or  survey  situations  which  have  a  structure  analogous  to 
the  typical  multi-way  factorial  (cross-classification)  design  fui  con¬ 
tinuous  data.  One  such  example  which  has  been  analyzed  a  number  of 
times  in  the  literature  (e.£.  Lombard  and  Doering  [1947],  Dyke  and 
Patterson  [1952],  Cox  and  Snell  [1968],  Bishop  [1969],  and  Goodman  [  1970] )  , 
appears  in  Table  5.2.1  and  corresponds  to  data  from  a  sample  of  177° 
individuals  who  were  cross  classified  according  to  the  following  five 
categorical  variables: 

1.  whether  they  read  newspapers  or  not 

2.  whether  they  lirtened  to  the  radio  or  not 

3.  whether  they  read  books  and  magazines  (solid  reading)  or  not 

4.  whether  they  attended  lectures  or  not 

5.  whether  their  knowledge  of  cancer  was  good  or  poor 
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Strictly  speaking,  this  is  s  "no  factor,  five  response"  situation 
since  only  the  sample  site  n  »  1729  is  fixed  and  for  each  individual 
all  classifications  are  random  variables.  On  the  other  hand,  the 
parameters  of  greatest  practical  interest  pertain  to  the  conditional 
probabilities  of  good  or  poor  knowledge  of  cancer,  given  fixed  cate¬ 
gories  for  exposure  to  newspapers,  radio,  solid  reading  and  lectures, 
this  conditional  point  of  view  allows  the  situation  to  be  considered  for 
a  ..lytical  purposes  in  term-,  of  a  "four  factor,  one  response"  model 
with  knowledge  of  cancer  being  the  response.  Hence,  the  data  are 
considered  to  represent  16  independent  binomial  samples  as  displayed 
In  the  16  x  2  array  in  columns  5  and  6  of  Table  5.2.1.  The  marginal 
total  for  each  row  is  assumed  to  be  fixed  and  the  data  from  different 
tows  are  assumed  to  be  statistically  independent.  These  assumptions 
are  essentially  ustified  by  the  conditional  mint  of  view.  The 
vector  of  condl tionsl  relative  frequencies  p  associated  with  good 
knowledge  of  cancer  in  the  various  newspaper  x  radio  x  solid  reading 
x  lectures  categories  appears  in  column  7  of  Table  5.2.1  and  the 
corresponding  estimated  variances  appear  in  column  8.  It  is  appropriate 
to  note  that  even  without  the  assumptions  associated  with  the  "four 
factor,  one  response"  model,  these  conditional  relative  frequencies 
are  uncorrelated  with  one  another.  Hence,  their  analysis  in  terms 
of  linear  models  and  weighted  least  squares  still  applies  independent 
of  the  conditional  point  of  view.  In  other  words,  the  only  aspect 
of  this  discussion  which  is  actually  "conditional"  is  the  set  of 
parameters  which  are  conditional  probabilities  of  good  knowledge 
of  cancer  as  related  to  exposure  to  various  media.  However,  this  is 
not  unreasonable  since  these  are  the  parameters  which  can  be  dealt 
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with  in  practical  terns  because  an  individual's  exposure  to  media 
can  be  controlled  like  a  factor  a,  priori  even  though  here  it  was 
measured  a  posteriori . 

Several  linear  models  were  fitted  to  the  set  of  conditional 
relative  frequencies  in  column  7  of  Table  5.2.1.  The  first  of  these 

was 


1111111111111111 
1  1  1  1  1  1  1  1  -1  -1  -1  -1  -I  -1  -1  -1 

1  1  1  1  -1  -1  -1  -1  1  1  1  1  -1  -1  -1  -1 

1  1  1  1  -1  -1  -1  -1  -1  -1  -1  -1  1  1  1  1 

i  l  -1  -1  1  1-1-1  1  1-1-1  1  1-1-1 

1  1-1-1  1  1  -l  -1  -1  -1  1  1-1-1  1  1 

1  1  -l  -1  -1  -1  1  1  1  1  -1  -1  -1  -1  1  1 

1  1  -1  -1  -1  -1  1  1-1-1  1  1  1  1-1-1 

1-1  1-1  1-1  1-1  1-1  1-1  1-1  1-1 
1-1  1-1  1-1  1-1-1  1-1  1-1  1-1  1 
1-1  1-1-1  1-1  1  1-1  1-1-1  1-1  1 
1-1  1-1-1  1-1  1-1  1-1  1  1-1  1-1 
1-1-1  1  1-1-1  1  1-1-1  1  1-1-1  1 
1-1-1  1  1-1-1  1-1  1  1-1-1  1  1-1 
1-1-1  1-1  1  1-1  1-1-1  1-1  1  1-1 
1  -1  -1  1-1  1  1-1-1  1  1-1  1  -1  -1  1 


(5.2.1) 


This  is  a  complete  model  since  the  number  of  effects 
columns  of  X^)  is  equal  to  the  number  of  distinct  factor  combinations 
(i^.e .  ,  tl.e  number  of  rows  of  X^).  A  description  of  the  effects 
corresponding  to  the  columns  of  together  with  significance  tests 
of  their  importance  appear  In  the  respective  rows  of  Table  5.2.2. 
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can  be  controlled  like  a  factor  _a  priori  even  though  here  It  was 
measured  a  posteriori. 

Several  linear  models  were  fitted  to  the  set  of  conditional 
relative  frequencies  in  column  7  of  Table  5.2.1.  The  first  of  these 
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This  Is  a  complete  model  since  the  number  of  effects  U-e.  , 
columns  of  X^)  Is  equal  to  the  number  of  distinct  factor  combinations 
(l.e . ,  the  number  of  rows  of  X^).  A  description  of  the  effects 
corresponding  to  the  columns  of  X^  together  with  significance  tests 
of  their  importance  appear  in  the  respective  rows  of  Table  5.2.2. 


495 


Table  5.2.2 


Source  of  Variation  D 

.F. 

X2 

- 

Nava pa par 

I 

17.94 

Radio 

1 

4.10 

Newspaper  x  Radio  Interaction 

1 

0.03 

Solid  Reading 

T 

4.78 

Nowapaper  x  Solid  Reading  Interaction 

1 

0.29 

Radio  x  Solid  Reading  Interaction 

1 

0.75 

Newspaper  x  Radio  x  Solid  Reading  Interaction 

1 

1.10 

Lectures 

1 

2.34 

Newspaper  x  Lectures  Interaction 

) 

3.44 

n 

Radio  x  Lectures  Interaction 

1 

1.13 

Newspapers  x  Radio  x  Lectures  interaction 

1 

0.12 

Solid  Reading  x  Lectures  Interaction 

1 

6.55 

Newspaper  x  Solid  Reading  x  Lectures  Interaction 

1 

1.28 

Radio  x  Solid  Reading  x  Lectures  Interaction 
Newspapers  x  Radio  x  Solid  Reading  x  Lectures 

1 

1.19 

Interaction 

1 

1.09 

Simultaneous  effects  of  all  sources 

15 

261.04 

The  analysis  In  Table  5.2.2  Is  analogous  to  that  for  a  2x2x2x2  factorial 
design.  It  indicates  that  there  are  significant  effects  due  to  newspapers 
(a  *  .01),  radio  (a  -.05),  solid  reading  (a  *  .05),  solid  reading  x 
lectures  Interaction  (a  -  .05),  and  newspaper  x  lecture  Interaction 
(a  •  .10).  However,  once  these  conclusions  have  been  noted,  the  next 
objective  Is  to  find  a  model  with  as  few  parameters  as  possible,  but 
which  still  accounts  for  most  of  the  variation  in  the  set  of  conditional 
reladve  frequencies  reflecting  good  knowledge  of  cancer.  The  results 
In  Table  5.2.2.  warn  the  analyst  that  he  must  pay  careful  attention  to 
the  Interaction  of  lectures  with  newspapers  and  solid  reading  in  for¬ 
mulating  this  model.  Some  insight  for  this  purpose  can  be  attained  by 
looking  at  Table  5.2.3  which  shows  the  effect  of  each  of  the  factors 
within  fixed  levels  of  the  other  three  factor0. 
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Table  5.2.3 


Read 

News¬ 

papers 

Listen 

to 

Radio 

Solid 

Read¬ 

ing 

Attend 

Lectures 

Newe- 

peper 

Effect 

Radio 

Effect 

Solid 

Reading 

Effect 

Lectures 

Effects 

Yea 

Yes 

Yes 

Yes 

0.49 

0.14 

0.07 

0.14 

Yes 

Yes 

Yes 

No 

0.10 

0.07 

0.23 

A 

Yea 

Yea 

No 

Yes 

0.10 

0.13 

A 

0.30 

Yes 

Yes 

No 

No 

0.16 

0.05 

A 

A 

Yes 

No 

Yes 

Yes 

0.33 

A 

0.06 

0.07 

Yes 

No 

Yes 

No 

o.os 

A 

0.21 

A 

Yes 

No 

No 

Yes 

0.37 

A 

A 

0.22 

Yes 

No 

No 

No 

0.14 

A 

A 

A 

No 

Yes 

Yes 

Yes 

* 

-0.02 

-0.32 

-0.25 

No 

Yes 

Yes 

No 

A 

0.05 

0.29 

A 

No 

Yes 

No 

Yes 

it 

0.40 

A 

0.36 

No 

Yes 

No 

No 

it 

0.03 

A 

A 

No 

No 

Yes 

Yes 

A 

A 

0.10 

-0.18 

No 

No 

Yes 

No 

A 

A 

0.27 

A 

No 

No 

No 

Yes 

A 

A 

A 

-0.01 

No 

No 

No 

No 

A 

A 

A 

A 

In  this  cable,  the  difference  between  the  proportion  with  good  knowledge 


and  the  proportion  with  poor  knowledge  for  a  given  factor  is  displayed 
for  every  combination  of  the  other  three  factors  in  those  rows  of  the 
table  for  which  the  given  factor  is  at  the  "yes"  value. 

From  Table  5.2.3,  it  is  apparent  that 

1.  In  all  cases,  newspapers  have  a  positive  effect  varying  from  moderate 
to  large  In  magnitude. 

2.  Radio,  for  the  ao9t  part,  has  a  positive  effect  varying  from  small 
to  large  in  magnitude. 

3.  Except  for  one  instance  where  it  showed  a  large  negative  effect, 
solid  reading  has  a  positive  effect  varying  from  small  to  large 
In  magnitude. 


4  97 


4.  The  affects  of  lectures  are  fairly  complex 


a.  For  newspaper  readers.  It  has  a  positive  effect  varying  from 
small  to  large  In  magnitude. 

b.  For  newspaper  non-readers  but  solid  readers.  It  has  a  moderate 
negative  effect. 

c.  For  newspaper  non-readers,  radio  listeners,  and  non-solid 
readers,  it  has  a  large  positive  effect. 

d.  For  newspaper  non-readers,  radio  non-listenera  and  non-solid 
readers,  it  has  essentially  no  effect. 


As  a  consequence  of  the  preceding  set  of  conclusions,  the  second  linear 
model  considered  was 
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0 

This  is  an  Incomplete  model  since  X^  has  8  <  16  columns.  Its  use 
must  be  justified  by  a  goodness  of  fit  test  which  measures  the  model's 
ability  to  account  for  the  variation  in  the  data.  The  test  statistic 
for  this  is  the  weighted  residual  sum  of  squares  shown  in  Table  5.2.4 
along  with  significance  tests  of  the  respective  effects  corresponding 
to  the  columns  of  X,. 
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Table  5.2.4 


Source  of  Variation 


D.F. _ X* 


Newspaper 

Radio 

Solid  Reading 

Lectures  within  Newspaper  Readers 
Lectures  within  Newspaper  Non-Readers. 

Radio  Listeners,  and  Solid  Re.-v’ 
Lectures  within  Newspaper  Non-Rcadet 

Listener* ,  and  Non-Solid  Reader* 
Lectures  within  Newspaper  Non-Readers, 
Non-Listenors ,  and  Solid  Readers 


Simultaneous  effects  of  above  sources 
Residual 


1 

37.47 

1 

7.21 

1 

44.88 

1 

7.46 

rs 

1 

0.63 

tadlo 

1 

1.58 

Radio 

1 

2.79 

7 

256.34 

8 

4.70 

Since  the  residual  sum  of  squares  Is  not  significant  (a  c  .25),  one 
may  say  that  the  model  (5.2.2)  provides  an  adequate  fit  to  the  data. 

In  addition,  significant  effects  are  detected  for  newspapers  (a  ■  .01), 
solid  read'ng  (a  "  .01),  and  lectures  within  newspaper  readers  (a  ■  .01). 
Although  the  other  effects  of  lectures  are  not  detected  as  significant 
(a  ■  .05),  it  is  not  really  appropriate  to  elimlnete  them  from  tlu 
model  because  the  following  estimates  of  the  effects  in  indicate 
that  they  are  essentially  of  the  same  magnitude  as  the  significant 
effects  of  lectures  within  newspaper  readers. 


Overall  Mean  0.424 

Newspaper  Effect  0.109 

Radio  Effect  0.041 

Solid  Reading  Effect  0.096 

Lectures  within  newspeper  readers  0.068 

Lectures  within  newspaper  non-readers,  radio 

listeners,  and  solid  readers  -0.069 

Lectures  within  newspaper  non-readers,  radio 

listeners,  and  non-solid  readers  0.072 

Lectures  within  newspaper  non-readers,  radio 

ron-listeners ,  and  solid  readers  -0.078 
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As  a  result,  the  third  model  considered  was 
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-1 

l 
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-l 
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i  -l 
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-i 

0 

which  ail  of  the  varying  lecture  effects  have  been  placed  into  a 
• -'.gle  complex  Interaction  term  which  reflects  the  different  direc¬ 
tions  the  effects  display  at  different  combinations  of  the  other 

2 

factors.  The  appropriate  X  -statistics  for  testing  the  effects  in 
the  model  (5.2.3)  are  given  in  Table  S.?.S. 


table  5.2.5. 


Source  of  Variation 

I).F. 

X 

Newspaper 

1 

46. 

.62 

Radio 

1 

9. 

,18 

Solid  Reading 

1 

54, 

.38 

Lectures  as  interacting  with  other  factors 

1 

12, 

.32 

Simultaneous  effects  of  above  sources 

4 

250, 

.30 

Residual 

11 

A . 

.7  U 

Since  the  residual  sum  of  squares  Is  not  significant  (a»  .25),  one 
may  say  that  the  model  (5.2.3)  provides  n  satisfactory  fit  to  the 


data.  Moreover,  all  of  the  effects  Included  in  thla  model  Arc  signlfi- 
cant  at  the  a  •  .01  level.  Finally,  estimates  for  the  various  effects 

are 


Overall  Mean  0.426 
Newspaper  Effect  0.109 
Radio  Effect  0.041 
Solid  Reading  Effect  0.097 
Lectures  as  interacting  with  other  factors  0.071 


Predicted  values  for  the  conditional  probabilit  es  of  good  knowledge 
of  cancer  within  each  factor  combination  arc  given  fci  the  model 
(5.2.3)  in  column  9  of  Table  5.2.1. 

Although  the  preceding  discussion  has  led  to  a  fairly  satis¬ 
factory  analysis,  some  additional  considerations  are  worth  noting. 

First  of  all,  although  the  residual  sum  of  squares  for  the  model  speci¬ 
fied  by  Xj  is  not  significant  (*  ■  .25)  when  compared  to  the  chi-square 
distribution  with  D.F.  »  11,  its  value  4.74  does  exceed  3.84  which  is 
the  u  •  .05  critical  point  of  the  chi-square  distribution  with  D.F.  «  l. 
Hence,  ic  is  possible  that  this  residual  conceals  a  significant  single 
degree  of  freedom  which  is  Important  in  understanding  the  variation  in 
the  data.  Whether  this  is  actually  the  case  or  not  is  difficult  to 
ascertain,  although  recognition  of  patterns  in  the  differences  between 
observed  relstive  frequencies  and  corresponding  predicted  values 
provides  some  guidance.  Alternatively,  one  can  proceed  to  fit  other 
models  until  a  reasonable  one  is  found  for  which  the  residual  sum  of 
squares  is  not  only  non-significant  but  does  not  exceed  3.84  and 
hence  cannot  conceal  a  significant  component.  For  the  data  in  this 
example,  such  a  model  was 
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1  1 
1  1 
1  1 
1  1 
1  -1 
1  -1 
1  -1 
1  “1 
1 


1  -1 
1-11 
1-11 
1  -1  1 
1  -1  -1 
1  -1  -1 
1  -1  -1 
1  -1  -1 


1  1 
1  -1 
-1  1 
-1  -1 
1  1 
1  -1 
-1  1 
-1  -1 
0  0 


0 

0 

0 

0 

0 

0 

0 

0 

0 

0 


0 
0 
0 
0 
0 
0 
0 
0 
-1 
1 
1 

-1 

0 

0  1  -1 
0-10 
0  -1  0 


(5.2.4) 


with  the  residual  sum  of  squares  X  -  2.57  with  D.F.  »  8.  Because  of 
certain  similar  patterns  In  magnitudes  of  effects  the  model  based  on 
was  modified  to 
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i  i  i  i 
111-1 
11-11 

i  l-i-i 
1-111 
i  -l  1-1 
l-i-i  i 
i  -l  -l  -l 


i  -l 
i  -l 
i  -i 
i  -l 
i  -i 


i  -l  -i 
i  -l  -i  -l 
i  -l  -l  -i 


0  0-1 
0  0  1 

0  0  1 

0  0-1 
1  -1  0 

l  1  0 

0  0 
0  0 


(5.2.5) 


by  the  same  reasoning  that  led  from  X,  to  X,.  The  appropriate  X  -  star,  is- 

-t  -  J 

tics  for  testing  the  effects  in  the  model  (5.2.5)  are  given  in  Table  5.2.6. 
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Tab la  5.2.6 


2 

Source  of  Variation _ D.F. _ X 


Newspapers 

1 

40.66 

Radio 

1 

7.63 

Solid  Reading  as  modified 

1 

56.51 

Lectures  as  modified 

1 

9.71 

Solid  Reading  x  Lecture  Interaction 

in 

Newspaper  non-readers  and 

radio 

listeners 

1 

11.37 

Simultaneous  effects  of  above 

sources 

5 

258.45 

Residual 

10 

2.59 

Hence,  this  model  provides  a  satisfactory  fit  to  the  data  and  all  of 


the  effects  included  in  it  are  significant  at  the  a  ■  .01  level. 

Estimates  for  the  various  effects  in  (5.2.5)  are 

Overall  Mean  0.424 

Newspaper  effect  0.109 

Radio  effect  0.039 

Solid  reading  as  modified  effect  0.100 

Lecture  as  modified  effect  0.070 

Solid  reading  x  lecture  interaction  in  news-  0.149 
paper  non-readers  and  radio  listeners 


Predicted  values  for  the  conditional  probabilities  of  good  knowledge 
of  cancer  bas^d  on  the  model  (5.2.5)  are  given  in  Column  10  of  Table 
5.2.1.  It  is  apparent  that  this  model  has  the  advantage  of  not  only 
accounting  for  the  obvious  complex  effects  of  lectures  but  the  more 
subtle  ones  due  to  solid  reading. 

Finally,  some  further  justification  for  the  objective  of 
finding  models  for  which  the  residual  sum  of  squares  does  not  exceed 
3.84  may  be  seen  in  noting  what  happens  when  a  simple  no  interaction 
model  is  fitted  to  the  data;  namely. 


503 


*6 


11111 
1111-1 
111-11 
1  1  1-1-1 
11-111 
1  1-1  1-1 

1  1-1-1  1 
1  1-1-1  -1 
1-1111 
1-1  1  1-1 

1-1  1-1  1 
1-1  1-1  -1 
1-1-1  1  1 
1-1-1  1  -1 
1  -1  -1  -1  1 
1  -1  -1  -1  -1 


(5.2.6) 


1 

4 


2 

The  X  -statistics  for  testing  the  effects  in  the  model  (5.2.6)  are 
given  in  Table  5.2.7. 


Table  5.2.7 


2 

Source  of  Variation  D.F.  X 


Newspapers 

1 

34. 

.37 

Radio 

1 

6. 

.08 

Solid  Reading 

1 

79. 

.24 

Lectures 

1 

4. 

.34 

Simultaneous  effects  of  above  sources 

4 

248. 

.32 

Residual 

11 

12. 

.72 

Estimates  for  the  various  effects  in  (5.2.6)  are 


Overall  mean  0.436 
Newspaper  effect  0.072 
Radio  effect  0.033 
Solid  reading  effect  0.111 
Lecture  effect  0.043 


and  predicted 
The  important 
is  definitely 


vaLues  based  on  these  appear  in  colinn 
point  to  note  here  is  that  the  :esidual 
not  significant  (a  -  .25)  when  compared 


11  of  Table  5.2.1. 
sum  of  squares 
with  the  chi-square 
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distribution  with  D.F.  ■  11.  Moreover,  from  the  discussion  of  the 
model  (5.2.1)  based  on  X^,  It  Is  apparent  that  this  residual  definitely 
conceals  the  significant  interactions  which  lecture  effects  have  with 
newspaper  and  solid  reading  effects.  One  consequence  of  this  is  the 
presence  of  fairly  lArge  residuals  for  several  of  the  factor  combina¬ 
tions. 

In  summary,  when  fitting  models  to  complex  categorical  data 
situations,  some  caution  must  be  exercised  in  interpreting  the  residual 
sum  of  squares  as  a  goodness  of  fit  test.  The  safest  criterion  Is  to 
require  that  this  residual  sum  of  squares  .ioe3  not  exceed  3.84  and 
hence,  does  not  include  any  individually  significant  degrees  of 
freedom.  When  this  goal  cannot  be  realistically  achieved,  then  effort 
should  be  directed  at  finding  a  model  for  which  the  residual  sum  of 
squares  Is  non-slgnlf leant  (when  tested  using  the  appropriate  D.F.) 
and  as  S'-?’1  a«  jio.  1h*c.  Further  justification  can  bo  given  to  such 
models  if  there  are  no  striking  patterns  in  the  differences  between 
observed  and  predicted  values.  Finally,  once  a  model  is  fitted,  it 
is  desirable  that  the  test  of  significance  on  each  separate  effect 
is  individually  significant  at  the  a  «  .05  level.  Otherwise,  an 
attempt  should  be  made  to  reduce  the  number  of  effects  in  the  model 
by  refining  their  definitions.  This  last  guideline  requires  careful 
judgement  because  all  tests  of  significance  on  the  parameters  of 
given  linear  model  are  adjusted  in  a  covariance  sense  for  all  effects 
which  are  not  specified  in  the  corresponding  hypotheses.  This 
Important  point  follows  from  the  fact  that  in  a  weighted  analysis, 
orthogonal  effects  are  no  longer  uncorrelated  and  hence  overlap  each 
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other  in  a  considerably  more  subtle  fashion  than  in  the  usual 

unweighted  analysis  of  variance  situation.  In  particular,  one  consequence 

of  this  which  is  readily  visible  is  that  in  each  of  the  tables  5.2.2- 

2 

5.2.7,  the  simultaneous  test  of  effects  has  an  X  -value  which  is  con- 

2 

slderably  larger  than  the  sum  of  the  X  -statistics  for  the  individual 
component  degrees  of  freedom.  Other  than  this  warning,  essentially 
the  only  other  advice  that  the  researcher  needs  is  that ,  since  this 
type  of  analysis  is  based  on  linear  models  and  weighted  least  squares, 
whatever  types  of  analysis  that  can  be  obtained  by  using  multiple 
regression  procedures  either  in  a  stepwise  fashion  in  the  sense  con¬ 
sidered  here  or  otherwise  apply  equally  well  for  categorical  data  as  for 
continuous  data.  A  further  discussion  of  this  topic  in  the  context 
of  another  example  is  given  in  Johnson  and  Koch  [1970a]. 

5.3  A  split-plot  contingency  table. 

In  this  section,  wc  shall  consider  an  example  of  a  categorical 
data  model  analogous  to  split-plot  experiments  in  analysis  of  variance. 
These  models  have  been  discussed  by  Koch  and  Reinfurt  [1970]  and  corre¬ 
spond  to  a  "uni-factor,  multi-response"  situation  with  the  following 
structure:  Within  each  sub-population  (whole  plot  factor),  each 
member  of  the  corresponding  group  of  subjects  Is  exposed  to  each  of 
the  d  levels  of  some  treatment  (split-plot  factor)  with  the  resulting 
responses  being  classified  into  one  of  s  levels.  The  resulting 
data  may  then  be  represented  inanrxsxsx  ...  x  s  contingency 
table  of  (d+1)  dimensions  where  r  denotes  the  number  of  sub¬ 
populations  (levels  of  the  whole-plot  factor)  and  s^  denotes  the  number 
of  possible  multivariate  responses.  The  dist inguishing  feature  of 

these  models  is  that  the  d  responses  are  measured  in  essentially 
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the  same  terms  and  factor-like  comparisons  across  them  are  of 
Interest.  Specifically,  the  hypotheses  of  interest  are 

1.  There  are  no  differences  among  the  sub-populations  with 
respect  to  the  distribution  of  the  d-variate  response 
vector  (i.e.  ,  there  are  no  whole-plot  factor  effects). 

2.  There  are  no  differences  among  the  effects  of  the  d 
levels  of  the  treatment  in  each  sub-population  (i..£.  , 
there  are  no  split-plot  factor  effects). 

3.  There  are  no  differences  across  sub-populations  with 
respect  to  differences  among  the  effects  of  the  d 
level3  of  the  treatment  (i_.e. ,  there  is  no  whole-plot 
x  split-plot  interaction). 

Certain  of  these  concepts  become  more  clear  if  the  s  response  categories 
may  be  quantitively  scaled.  In  this  case,  hypotheses  can  be  formu¬ 
lated  in  terms  of  mean  scores  for  each  sub-population  x  treatment 
combination  (where  the  d  mean  scores  within  a  given  sub-population 
are  correlated  with  one  another  since  each  is  based  on  the  same  set 
of  subjects).  As  a  result,  test  statistics  for  hypotheses  pertaining 

to  treatment  effects  (split-plot  factor)  are  analogous  to  Hotelling 

2 

T  procedures  while  the  others  have  more  complex  analogues  in  multi¬ 
variate  analysis  of  variance  (see  Cole  and  Grizzle] 1966)  and  Koch  llJ69]). 

One  type  of  data  which  can  be  interpreted  from  the  point  of 
view  of  split-plot  contingency  table  models  are  social  mobility  tables 
for  different  countries.  In  Tabic  5.3.1  ,  data  which  were  recently 
described  in  Mosteller  (1968]  are  presented.  These  data  originally 
arose  from  Glass's  British  study  and  svaiastoga's  bullish  study  c.f 
occupational  mobility.  The  frequencies  in  the  table  correspond  to 
the  Joint  distributions  of  father's  and  son's  occupation  on  a  five 
point  ordinal  scale.  These  data  may  be  regarded  as  arising  from  a 
"one  factor,  two  response"  split-plot  contingency  table  model  in 
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which  country  Is  tne  whole-plot  factor  and  father  va.  son  Is  ths 
split-plot  factor. 

Table  5.3.1 


British 

Son's 

Occupation 

Status 

Data 

1 

2 

3 

4 

5 

Total 

1 

50 

45 

8 

18 

8 

129 

2 

28 

174 

84 

154 

55 

495 

Father's 

Occupation 

3 

11 

78 

110 

223 

96 

518 

Status 

4 

14 

150 

185 

714 

447 

1510 

s  ; 

3 

42 

72 

320 

411 

848 

Total 

1U6 

489 

459 

1429 

1017 

r _ _ 

Danish 

Data 

i 

18 

17 

16 

4 

2 

57 

2 

24 

105 

109 

59 

21 

318 

Father's 

Occupation 

3 

23 

84 

289 

217 

95 

708 

Status 

4 

8 

49 

175 

348 

198 

1 

|  778 

6  8  69  201  246 

79  263  658  829  562 


On  applying  the  A-matrix 


b 


11111  22222 
12345  12345 


33333  44444  55555 
12345  12345  12345 


Co  the  sub-table  corresponding  to  each  country  arranged  in  a  single 
row  vector  by  placing  the  existing  rows  one  after  the  other,  mean 
status  scores  for  father's  and  son 'a  occupation  within  each  country 
may  be  determined.  The  resulting  estimates  and  the  corresponding 
estimated  variance-covariance  matrix  are  given  in  Table  5.3.2. 


Table  5.3.2 


Country 

Person 

Mean 

Score 

r* 

Brit. 

Father 

.ariance  with 
Brit .  1 

Son  ! 

Score  for: 

Den. 

Father 

Den. 

Son 

Britain 

Father 

3.70 

0.00034 

0.00015 

Britain 

Son 

3.79 

0.00015 

0.00035 

Denmark 

Father 

3.59 

0.00046 

0.00022 

Denmark 

Son 

3.64 

0.00022 

0.00047 

where  the  "blank"  covariances  are  zero  because  the  data  from  the  two 
separate  countries  Involve  different  subjects  and  hence  are  independent 
of  each  other.  If  the  linear  model  corresponding  to  the  design  matrix 


1  1  1 


X 


11-1 

1-11 


1 


-1  -1 


(3.3.1) 
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I 

i 

ji 


i 

i 

1 

I 


is  fitted  to  the  seen  score*  in  Table  5.3.2  by  weighted  least 

squares i  test  statistics  for  hypotheses  pertaining  to  the  effects  of 

country  and  father  vs.  son  and  thslr  interaction  may  be  teated  accord- 

2 

ing  to  the  mathodology  outlined  in  Section  3.  The  appropriate  X  - 
statistics  are  given  in  Table  5.3.3. 

Table  5.3.3 

2 

Source  of  Variation _ D.F. _ X 

1  28.78 

1  24.68 

Residual:  Interaction  of  Country 

and  Father  vs.  Son  1  1.46 

Hence,  it  Is  apparent  that  there  is  no  interaction  between  the  effects 

of  country  and  father  vs.  son  and  that  the  mean  occupation  status  score 

is  significantly  greater  for  Britain  than  for  Denmark  and  significantly 

greater  for  sons  than  for  fathers.  Finally,  the  estimates  for  the 

parameters  in  the  linear  model  defined  by  (5.3.1)  are 

Overall  Mean  -  3.680 
Country  Effect  *  0.065 
Father  vs.  Son  Effect  “  0.036 

These  estimates  can  be  used  to  determine  predicted  values  for  mean 
occupation  status  scores  for  fathers  and  sons  in  the  two  countries. 

In  the  preceding  discussion,  rhp  data  were  viewed  as  a  "one 
factor,  two  response"  situation  with  comparisons  of  country  effects 
and  father  vs.  son  effects  being  the  topics  of  primary  interest. 
Alternatively,  if  there  is  more  interest  in  studying  the  way  in  which 
the  mean  status  score  for  son's  occupstlon  depends  on  country  and  the 
corresponding  level  of  father's  occupation,  it  is  more  appropriate  to 
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Count  ry 

Father  vs.  Son 


J 

4 


a 


regard  the  data  in  terms  of  a  "two  factor,  one  response"  model  in 
which  son'a  occupation  la  the  response,  and  country  and  father's 
occupation  are  the  factors.  In  this  case,  the  A-matrix 

A  -  [1  2  3  4  5] 


is  applied  to  each  row  of  Table  5.3.1  (except  the  "total"  rows)  to 
determine  mean  scores  for  each  country  x  father's  occupation  status 
combination.  These  scores  together  with  their  corresponding  estimated 
variances  are  given  In  columns  3  and  4  respectively  of  Table  5.3.4. 

It  can  be  noted  that  these  mean  scores  are  uncorrelated  with  one 
another  since  they  are  based  on  the  conditional  distributions  of  son's 
occupation  status  given  father's  occupation  status.  Three  linear 
models  were  fitted  to  the  set  of  mean  scores  for  son's  occupation 
status.  The  first  of  these  was 


1 

1 

.0 

•> 

4 

-8 

16 

1 

1 

-1 

1 

-1 

1 

1 

1 

0 

0 

0 

0 

1 

1 

1 

1 

1 

1 

1 

1 

2 

4 

8 

16 

1 

-l 

-2 

4 

-8 

16 

1 

-l 

1 

-1 

1 

1 

-1 

0 

0 

0 

0 

1 

-1 

1 

1 

1 

1 

1 

-1 

2 

u 

8 

16 

where  the  status  scores  for  father's  occupation  are  being  treated, 

for  simplicity,  as  lying  on  an  equally  spaced  scale  centered  at  the 

2 

middle  level  of  "3".  The  appropriate  X  -statistics  for  testing  the 
effects  in  the  model  (5.3.2)  are  given  in  Table  5.3.5. 
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iA  Aiti! 


Table  5.3.S 


Country  1  15.33 

Father's  Occupation  Status  4  14 '6. 94 


Residual:  Interaction  of  Country  with  _  '  - 

Father's  Occupation  Status  *  ..'.-21.31 


v  , 

Since  the  Interaction  of  the  two  sources  of  variatioi\t'jL  s 'gnl'ican.  , 

4 

(a  ■  .05),  the  model  corresponding  to  X^  does  not  quit-*  <c-,.roprlate  .y 
fit  the  data  although  it  does  explain  a  substantial  proportion  of  the 


Only  linear  and  quadratic  components  of  father's  occupation  status 

together  with  their  interactions  with  country  were  included  in  (5.3.3) 

because  for  the  model  in  (5.3.2)  a  test  of  significance  for  the 

2 

cubic  and  quart lc  components  had  X  "3.51  with  D.F.  •  2.  From  this 
result  and  the  fact  that  such  effects  do  not  have  any  real  practical 
Interpretation  for  such  data  showing  a  monotone  relationship,  nelrher 

the  second  nor  the  third  models  considered  here  will  include  cubic  or 

2 

quartic  terms.  The  appropriate  X  -statistics  for  testing  the  effects 
in  the  model  (5.3.3)  are  given  in  Table  5.3.6.  Hence,  the  model  does 
fit  the  data  since  the  residual  is  not  significant  (a  »  .05^. 
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Table  5.3.6 


Source  of  Variation 


D.F. 


Country 

1 

20.10 

Linear  Component  of  Father's  Status 

1 

1005.48 

Quadratic  Component  of  Father's  Status 

1 

25.62 

Country  x  Linear  Component  Interaction 

l 

0.93 

Country  x  Quadratic  Component  Interaction 

1 

1.65 

Residual 

4 

7.78 

In  addition.  It  appears  as  If  neither  interaction  term  Is  significant 
at  the  a  “  .05  level.  However,  it  must  be  borne  in  mind  that  with 
weighted  least  squares,  apparently  orthogonal  effects  are  no  longer 
uncorrelated.  Hence,  all  tests  reflect  the  contribution  of  a  given 
effect  conditional  on  all  other  effects  being  in  the  model.  Thus 
since  the  model  based  on  X^  tcld  us  that  interaction  was  present,  the 
primary  conclusion  from  Table  5.3.6  is  that  this  interaction  can  be 
explained  in  terms  of  essentially  one  component,  i_.e.  ,  cither  the 
country  x  linear  component  or  the  country  x  quadratic  component.  The 
estimates  of  the  parameters  in  the  model  X^  defined  by  (5.3.3)  are 


Overall  Mean  3.476 
Country  Effect  0.083 
Linear  Component  of  Father's  Status  0.500 
Quadratic  Component  of  Father's  Status  -0,052 
Country  x  Linear  Component  Interaction  -0.015 
Country  x  Quadratic  Component  Interaction  -0.013 


Since  the  country  x  quadratic  component  interaction  has  a  larger 
relative  magnitude  with  recpect  to  the  quadratic  component  main 
effect  than  the  country  x  linear  component  interaction  has  relative 
to  the  linear  component  main  effect,  the  third  model  considered  is 
equivalent  to  that  defined  by  with  the  country  x  linear  component 
interaction  eliminated.  This  model  was 
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X,3 


11-240 
11-110 
110  0  0 
11110 
112  4  0 
1-1-2  0  4 

1-1-1  0  1 
1-1000 
1-1101 
1-1204 


(5.3.4) 


2 

The  Appropriate  X  -statistics  for  testing  the  effects  in  the  model 
(5.3.4)  are  given  in  Table  5.3.7.  First  it  can  be  noted  that  the 
model  does  fit  the  data  since  the  residual  is  not  significant  (for  a  ■  .05^ 
In  addition,  every  effect  is  individually  significant  (a  *  .05).  Finally, 
the  hypothesis  of  equality  of  the  two  separate 


Table  5.3.7 


Source  of  Variation 


D.F.  X2 


Country  ] 
Linear  Component  of  Father's  Status  1 
Quadratic  Component  of  Father's  Status  in  Britain  1 
Quadratic  Component  of  Father's  Status  in  Denmark  1 


19.49 

1033.90 

32.33 

5.21 


Residual 


5  8.71 


2 

quadratic  effects  can  be  tested  with  the  resulting  X  ■  6.11  where 
D.F.  *  1  implying  that  there  is  an  interaction  here  which  is  signifi¬ 
cant  (a  ■  .05).  The  estimates  of  the  effects  in  the  model  specified 
by  (5.3.4)  are 


Overall  Mean  3.475 

Country  Effect  0.081 

Linear  Component  of  Fattier 's  Status  0.500 

Quadratic  Component  of  Father's  Status 

in  Britain  -0.070 

Quadratic  Component  of  Father's  Status 

in  Denmark  -C.03I 
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These  estimates  to  some  extent  show  the  somewhat  different  relation¬ 
ships  between  son's  occupation  status  and  father's  occupat : or  status 
in  the  two  countries.  Finally,  predicted  values  for  the  mean  scores 
for  son's  occupation  status  for  each  country  x  father's  occupation 
status  combination  are  given  for  the  linear  models  based  on  X^,  X^, 

Xj  in  columns  5,  6,  7,  respectively,  of  Table  5.3.4.  In  comparing  ob¬ 
served  and  predicted  values,  it  should  be  noted  that  the  criteria  for 
weighted  analyses  forces  the  residuals  to  be  smallest  where  the  sample 
sises  are  the  largest. 

5.4  Drug  Example  wfth  Supplemented  Margins 

This  is  an  hypothetical  example  of  a  categorical  d;.ta  model  with 
supplemental  information  on  both  margins.  Suppose  a  certain  pharmaceu¬ 
tical  company  wished  to  compare  the  effects  of  drugs  A  and  B.  Of  the 
80  patients  available  for  study,  50  patients  (Group  1)  received  both 
drugs  under  essentially  the  sase  conditions  including  a  suitable  rime 
delay  between  applications  (with  the  order  of  presentation  of  A  and  rs 
random);  16  patients  (Group  11)  received  drug  A  only;  and  14  patients  j 

(Group  III)  received  drug  B  only.  In  each  case,  the  patient's  response  1 

to  the  particular  drug  was  observed  and  recorded  as  "none",  "slight",  1 

or  "strong".  The  results  are  given  in  Table  5.4.1.  The  main  question  -• 

i 

i 

of  interest  is  whether  the  distribution  of  strength  of  response  is  the  ; 

same  for  both  drugs. 


4 
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Table  5.4.1 


(Group  I) 

Response  to  A 

None 

(1) 

Response 

Slight 

(2) 

to  B 
Strong 
(3) 

Sub¬ 

total 

T - 

i  Response 
i  to  A  only 
j  (Group  11] 

Total 

None  (1) 

B 

6 

0 

10 

[  4 

14 

Slight  (2) 

H 

8 

9 

21 

j  7 

28 

Strong  (3) 

0 

5 

14 

19 

!  5 

24 

Sub-total 

8 

19 

23 

■m 

66 

Response  to  B  only 
(Group  III] 

3 

4 

*7 

l  ~ 

1 

14  : 

. 

i 

Total 

11 

23 

30 

* 

64  ! 

» 

j  80 

J_ _ 

Within  this  framework,  lot 

?'  *  ^n*  ni2’  ^13 ’  "21*  t,22’  ^23 ’  *31  ’  *37’  T 33* 
n’. 


<n\  n'(B),  n'u))  -  <4  ,6 ,0 ,4 ,8 ,9 ,0 ,5 ,14  ;4 , 7 , 5  ; 3 .4 , 7 ) 


Tor  this  example,  it  appears  reasonable  to  assign  scores  to  the  three 
levels  of  the  two  responses  and  to  test  for  equality  of  mea  •  scores 
over  the  two  marginals.  Namely 


V  aA“aB 

where  a  and  a  are  the  mean  scores  for  the  responses  to  drugs  A  and 
A  ft 

B,  respectively.  Therefore,  assigning  scores  a^  =  1  ,  a,  =  2,  and  a^  =  3 
to  the  response  categories  for  both  drugs  A  and  B  and  using  initial 
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with 


(5.4.6) 


Finally,  a  test  with  D.F.  »  1  for  (5.4.1)  is  obtained  ras  in  ..rlzrlc 
et  al,  [1969])  using  c x  and  a  from  the  first  stage  with  the  contrast 

D 

matrix,  C  *  (1,  -1).  Alternatively,  if  the  linear  model 


E<§)  -  X2h  where  X 


(5.4.7) 


is  fitted  using  weighted  least  squares,  the  residual  sum  of  squares 
provides  ttie  required  test  statistic  with  D.F.  ■  1. 

It  should  be  noted  that,  tor  the  test  uf  equality  of  m<-an  scores 
over  the  marginals,  empty  cells  are  no  problem  since,  as  long  as  the 
marginal  totals  are  non-zero,  the  required  estimate  of  the  covariance 
matrix  is  non-singular. 

The  usual  format  of  the  linear  model  approach  can  be  modified 
to  produce  this  analysis  as  follows.  First  the  data  are  considered 
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as  having  arisen  from  three  populations  (Groups  1,  II,  anu  III).  Thus 
the  following  matrices  comprise  the  basic  input. 


46048905  14 
n  -  47500000  0 
3x9  30040070  0 


r 

111222333000000000000000000 

123123123000000000000000000 

000000000123000000000000000 

000000000000000000100200300 

L. 


*2 

4x27 


The  model  as  given  by  (5.4.2)  fits  the  data  (i_.e.  ,  the  test  oc 

fit  gives  xl  “  0.30  with  D.F.  «  2).  io  obtain  the  test  in  (5.4.1), 
r 

namely,  H  :  a  •=  a  ,  use  X,  as  given  by  (5.4.5)  and  C  «  (1,  -1). 

Q  A  o  -1  ‘  — 

The  corresponding  test  statistic  with  D.F.  *  1  is  found  to  be 
2 

■  2.1  indicating  that  the  mean  scores  for  drugs  A  and  B  are  the 
same. 


5.5  A  sample  survey  situation 

In  this  section,  we  shall  briefly  consider  an  example  of 
categorical  data  obtained  from  a  sample  survey  situation.  These 
data  arose  from  a  survey  of  drivers  born  in  1050  and  licensed  in 
North  Carolina  in  1969.  The  sample  design  was  stratified  with  the 
three  strata  and  the  corresponding  sampling  rates  being 


Stratum 

White  Male 
White  Female 
Non-White  Male 


Sample  Size 

1034 

994 

1300 


Sampl ing  Rate 

(1/30) 

(1/25) 

(1/S) 


For  all  drivers  in  the  sample,  information  was  obtained  from  his 
driver  record  pertaining  to  numbers  of  violations  and  accidents.  In 
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addition,  the  drivers  were  sent  a  questionnaire  concerned  with  their 
exposure  to  risk  (e.£. ,  mileage,  night  driving,  etc.)  wnich  they  either 
returned  lanediately,  after  follow— ip,  or  not  at  all.  Hence,  there 
is  interest  in  whether  those  who  returned  the  questionnaire  tended 
to  hnve  better  driver  records  than  those  who  did  not.  As  a  result, 
the  data  in  Table  5.5.1  indicate  the  joint  distribution  of  the  responses 
pertaining  to  questionnaire  return  or  not  and  at  least  one  accident 
or  not  for  each  of  the  three  strata. 


Table  5.5.1 


Quest . 

Returned 

Quest . 

Not  Returned 

No  Acc . 

1+  Acc. 

No  Acc  . 

1+  Acc . 

White  Males 

429 

194 

269 

142 

White  Females 

529 

111 

303 

51 

Non-White  Males 

535 

165 

460 

140 

In  this  analysis,  attention  is  focused  on  the  proportion  of  no 
accident  drivers  within  each  questionnaire  return  status  x  stratum 
combination.  The  estimates  of  these  proportions  are  obtained  in  the 
usual  way  and  are  given  in  Table  5.5.2.  However,  the  variances  are 
adjusted  in  the  sense  of  being  multiplied  by  the  appropriate  finite 
population  correction  factor  (i.-e-,  bv  (.  1  —  F ^ >  where  is  the  sampling 
rate).  These  nro  also  given  in  Table  5.5.2, 
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Table  5.5.2 


Stratum 

Quest. 

Prop. 

Unad J . 

Finite 

AdJ  . 

Returned 

No .  Acc . 

Eat. 

Pop.  Corr. 

Est. 

Variance 

Veriance 

White  Hales 

yes 

0.69 

0.000346 

(29/30) 

0.000333 

White  Hales 

no 

0.65 

0.000550 

(29/30) 

0.000532 

White  Females 

yea 

0.83 

0.000224 

(24/25) 

0.000215 

White  Females 

no 

0.86 

0.000348 

(24/25) 

0.000334 

Non-White  Melee 

yea 

0.76 

0.000257 

(7/8) 

0.000225 

Non-White  Males 

no 

0.77 

0.000298 

(7/8) 

0.000261 

Once  the  estimated  variances  have  been  adjusted  for  finite  population 
sampling,  the  usual  analysis  applies.  Hence,  the  linear  model 


110  1 
X  -  110-1 

-  10  11 
10  1-1 
1-1-1  1 
_1  -1  -1  -1  _ 

was  fitted  to  the  data.  Test  statistics  for  the  effects  in  X  appear 
in  Table  5.5.3. 


Table  55.3 


Source  of  Variation 

P.F. 

X 

Strata 

•7 

78.61 

Questionnaire  Returned  or  Not 

1 

0.08 

Residual:  Interaction 

2 

2.98 

Hence,  it  appear  that  there  are  little  if  any  differences  among 
those  drivers  who  returned  the  questionnaire  and  those  who  did  not 
with  re spec t  to  the  proportion  of  no  accidents.  There  are  definite 
differences  among  the  strata. 
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ESTIMATION  IN  TRUNCATED  POISSON  DISTRIBUTIONS 

WITH  CONCOMITANT  EXPOSURE  INTERVALS  AND  TRUNCATION  POINTS 

A.  Clifford  Cohen 
The  Uni  vers ity  of  Ueorgia 

1.  INTRODUCTION 

Estimation  in  the  truncated  Poisson  distribution  has  previously 
been  considered  by  numerous  writers  including  Bliss  (1),  David  and 
Johnson  (4],  Moore  [6]  ,  Plackett  [7],  Subrahmanian  |8|,  the  writer 
and  others.  Our  concern  here  is  with  estimation  of  the  Poisson 
parameter  when  the  i—  observation  of  the  discrete  random  variable  X 
extends  over  a  specified  exposure  interval  of  magnitude  t.,  subject  to 
the  restriction  that  x.  ^_c.,  where  c.  is  a  specified  truncation  point. 
More  specifically,  our  attention  is  directed  toward  maximum  likelihood 
estimation  based  on  a  random  sample  consisting  of  n  independent 
observations  {x.,t^,c.},  where  x.  is  the  number  of  occurrences  of  the 
event  of  interest  observed  during  the  i—  interval  of  exposure,  while 
t^  and  Cj  are  corresponding  concomitant  values.  Of  course,  x.  and  c. 
must  be  non-negative  integers,  but  t;  is  permitted  to  assume  any 
positive  value.  Accident  data  in  which  reports  are  rendered  for  time 
intervals  of  varying  sice  with  truncation  as  stipulated  fall  into  the 
category  of  samples  under  consideration  here.  Inspection  data  in  which 
defects  per  unit  are  reported  for  units  of  differing  sices  might  also 
result  in  samples  of  the  type  considered  here. 

This  paper  has  been  reproduced  photographically  from  the  author's 
manuscript. 
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With  concomitant  values  t.  am!  c.  given,  the  applicable  I'oisson 
probability  function  may  be  written  as 


I’r  i  \  =  x .  1 1 .  ,e.  ; 
i'ii 

where 


1  -  1  { c  .-11.  C) 


MAXIMUM  1. (  KliLl  11001  •  hSTIMATION  Of  A 
The  likelihood  function  for  a  sample  consist  ini',  of  the  n  observed 
values  {x.,t.,c..l  for  which  the  prohalulitv  function  (11  is  .pul icahlc, 
follows  ;is 

-  •  t  .  \  . 

n  e  1  (  •  t  .  '  1 

i  =  1  \  .  f(  c  1 

i  i 
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might  also  he  employed  in  the  calculation  of  ■. 

3.  SOML  SPhCl.M  CAS!  S 

By  appropriately  restricting  the  values  of  t.  and  in  ((>) ,  we 
obtain  results  that  apply  in  various  case'  of  special  interest. 
Certain  of  these  specialized  results  are.  of  course,  quite  well  known. 

No  Truncation  —  Unequal  Intervals 

In  this  case,  c.  =  0  for  all  i.  and  equation  1 1» j  leads  to  the 
estimator 

n  n 


No  Truncation  —  F.qual  Intervals 

'f  in  addition  to  the  restriction,  c.  =  0  tor  all  i ,  we  impose  the 
further  restriction  that  t  ^  =  t  for  aii  i,  Cue  appi  liable  estimator 
reduces  to 

'•  1  */t  ,  (8 

__  n 

where  x  =  w  x./n.  When  t  -  1,  as  in  the  usual  case  of  estimation  in  the 

1  1 

Poisson  distribution,  then  C )  leads  to  the  familiar  estimator  =  7. 


Only  Zero  Class  Truncated  In  Making  l.acli  Observation  -  Unequal  Intervals 
In  this  case  *  1  tor  each  i,  and  the  estimating  equation  (b) 
reduces  to 


n  ii  - '  t . 

:  X  --  v.  ItVd-e  ’l|  ,  (91 

1  I  1 
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a  result  which  was  obtained  independently  by  one  of  the  writer's 
graduate  assistants,  Mr.  S.  S.  Srivastava. 

Only  Zero  Class  Truncated  In  Making  bach  Observation  —  I'.gual  Intervals 
When  we  further  specialize  the  proceeding  result  by  requiring  that 
t  *  t  for  all  i,  then  (9)  becomes 

7  =  >t/li  c  T|  .  no) 

If  we  specialize  still  further  by  requiring  that  t  =  1 ,  then  (10) 
reduces  to 

7  «  >./[l-e"|  .  (11) 

as  previously  given  in  (j)  ,  where  a  table  of  the  function  on  the  right 
side  of  (11)  is  providcJ  in  order  to  facilitate  solution  for  the  required 
estimate  x. 


4.  VARIANCI.  01  I  S  I  l MAT!  S 
The  asymptotic  variance  of  V  can  be  expressed  as 


V  (O  »  * 


(12) 


for  each  of  the  cases  considered  here.  The  second  derivative  in  the 
most  general  case  under  consideration  follows  from  (4)  as 


?2ln  L 

L  x. 

1  1  -  it* 

1 

f(c.-?l  -  f(e.-l) 

i  i 

tfbyn 

'  ‘ 

V2  1* 

I!' 

r. 

^  l 

1 

rfc  ) 

L  1 

i 

(13) 
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a  result  which  was  previously  given  in  |J|.  Kith  the  aid  of  tables 
such  as  those  of  Molina  [5],  this  derivative  can  easily  be  evaluated 
once  (0)  has  been  solved  for  The  variance  then  follows  from  (12). 

S.  AN  ILLUSTRATIVi:  l-XAMPl.l: 

In  order  to  illustrate  the  practical  application  of  estimators 
derived  here,  we  consider  inspection  results  on  the  number  of  surface 
defects  per  unit  found  in  products  of  differing  surface  areas  but 
which  were  manufactured  on  <the  same  production  line.  In  si’uv  of  the 
inspections,  no  truncation  takes  place.  In  others,  the  zeros  are 
eliminated,  and  in  still  others,  both  zeros  and  ones  arc  eliminated. 
Based  on  the  total  sample  which  results  from  combining  the  several 
sub-samples  involved,  wc  proceed  to  estimate  the  Poisson  parameter  X 
which  characterizes  the  underlying  common  production  process,  and  to 
determine  the  asymptotic  variance  of  this  estimate.  The  total  sample 
available  for  this  purpose  consists  of  inspection  results  on  222 
separate  manufactured  units.  These  data  arc  presented  in  fable  1,  with 
the  total  sample  broken  down  into  eight  sub- samples  having  values  of 
(t,c)  as  follows:  (1 ,0)  ,  (1 .5, 1 ) .  (2, 1)  ,  (.2.5, 1) ,  (3 . 1) ,  ( 1 ,2)  ,  (2 ,2) ,  and 
(3,2).  The  value  t  =  1  indicates  a  surface  area  of  one  unit,  while 
t  =  2  indicates  a  surface  area  of  two  units,  etc.  F;or  the  total 

number  of  defects  found  in  the  422  units  inspected,  we  have 

422  422 

E  x.  =  1S97.  For  the  sum  of  the  t's,  we  have  E  t.  =  100(1) 

1  1  1  1 

♦  94(1.5)  +  50(2)  ♦  32(2.5)  +28(3)  +  30(1)  +  34(2)  +  54(3)  =  7or  . 

These  values  are  substituted  into  ((>)  which  must  then  be  solvi  ..  -.or 
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the  required  estimate  >.  As  a  first  approximation,  wc  try  »  2.000U, 
which  comes  from  the  sub-sample  of  site  1 00  with  t  *  l  and  c  -  t>.  In 
general,  satisfactory  first  approximations  might  be  obtained  from  any¬ 
one  or  a  combination  of  the  available  sub-samples.  In  some  circum¬ 
stances,  a  judicious  guess  might  oven  suffice,  he  subsequently  arrive 
at  the  final  estimate  x  «  2.005,  by  linear  interpolation  as  follows 

_ f-m 

lfcb3.',2 
422 

L'  x.  «  1597.00 

1  ' 

1S"3.22 


» 

- r — 

2 . 100 

2.005 

2.000 


where  G(v)  has  been  written  for  the  right  side  of  the  estimating 
equation  (6).  1-or  the  variance  of  the  above  estimate,  we  employ (12) 
and  (13)  to  calculate  V(>)  -  0.0028. 

It  is  of  interest  to  note  that  the  upper  hound  on  '  as  given 
hy  (6a)  for  this  example  becomes 

*UB  =  l-c,97,:65  =  2.0i.-u  , 

a  result  which  further  justifies  the  selection  of  \j  *  2.000  as  our 
first  approximation. 
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MYTHS  IN  THE  DESIGN  OF  MEASUREMENTS 


XMCi  A,  Pullen,  Jr. 

U.S.  Array  Aberdeen  Research  and  Development  Cencer 
Ballistic  Research  Laboratories 
Aberdeen  Proving  Ground,  Maryland 

SUMMARY .  There  are  at  least  two  different  theories  which  must  be  taken  into 
account  in  the  design  of  experiments  for  asking  of  aeaeuraenta,  namely,  the 
theories  of  statistics,  and  the  theories  of  fusty  seta.  In  addition,  there 
are  aoae  Important  basic  principles  which  aust  be  taken  Into  account  in  the 
use  of  either  of  the  basic  theories. 

The  purpose  of  this  paper  is  to  explore  aoae  of  the  gray  areas  created  by 
the  mingling  of  the  two  theories  and  by  the  lack  of  ideal  conditions  which 
■aust  be  accepted  with  the  basic  theories  by  theaselvea.  Such  questions  us 
small  differences,  and  the  unavoidable  biases  in  supposedly  random  data  will 
be  considered. 

Further,  the  information  engineering  of  experiaents  to  assure  that  meaningful 
deterministic  parameters  are  given  proper  weighting  in  the  design  of  experi¬ 
ments  will  be  discussed. 

INTRODUCTION.  The  basis  for  measurement  of  scientific  data  required  in  any 
kind  of  engineering  design  is  intimately  dependent  on  the  theory  of  di  sign  of 
experiments.  In  fact,  the  proper  structure  for  an  experiment  for  making 
measurements  on  which  design  aust  be  based  is  dependent  on  a  diverse  set 
of  disciplines,  some  of  the  more  Important  being: 

1.  The  theory  of  etatistics 

2.  The  theory  of  furry  sets 

3.  The  theory  of  Greco-Roman  squares 

4.  Principles  of  information  engineering 

5.  General  mathematical  limitations 

Interestingly i  it  is  important  to  examine  NOT  ONLY  conventional  theory  of 
these  disciplines  as  normally  developed,  but  it  is  possible  even  more  Impor¬ 
tant  that  the  study  be  enlarged  in  each  case  to  determine  the  consequences 
of  built-in  biases. 

MYTHS  DUE  TO  STATISTICAL  BIAS.  The  theory  of  statistical  analysis  is  usually 
invoked  when  errors  of  a  random  nature  can  be  expected  to  be  encountered  which 
render  the  making  of  deterri.ui.atlc  evaluations  of  precision  difficult  if  not 
Impossible.  This  assumption  of  the  use  of  statistical  methods (particularly 
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Gaussian  atatlatl.es)  for  handling  arrora  poatulatas  that  there  la  a  vary  large 
collection  of  different  error  sources,  each  of  vhlch  haa  approximately  equal 
probability  of  occurrence.  In  tact,  thia  collection  of  errora  behavea  some¬ 
thing  like  a  multi-dimensions!  random-walk  problem.  (The  magnitudes  of  the 
random  stepa  and  their  probabilities  of  occurrence  are  by  Implication  equal.) 
Clearly,  the  validity  of  the  statistical  "sharpening"  of  the  final  results  la 
intimately  dependent  on  the  validity  of  the  implication  of  equality  of  the 
probabilities. 

Statistical  processes,  of  course,  are  not  limited  to  Gaussian  processes 
baaed  on  exp  (-h?  **),  but  can  include  various  kinds  of  skew  distributions  and 
other  shaped  distributions.  In  this  respect,  it  ia  Important  to  note  that 
If  one  attempts  to  develop  a  statistical  weighting  system  where  the  weight¬ 
ing  of  the  data  points  converges .  one  obtains  a  divergent,  and  hence  useless, 
statistical  system.  The  obvious  conclusion  chat  this  leads  to  is  that  the 
discarding  of  data  point  SOLELY  on  the  basis  that  they  differ  by  more  than 
some  number  of  sigma  units  from  the  balance  of  the  set  le  not  permissible. 

The  discard  of  data  points  can  only  be  justified  IF  there  are  known  physical 
or  other  reasons  which  throw  the  data  themselves  into  legitimate  question! 

It  Is  generally  assumed  that  en  Increase  in  the  number  of  date  point*  leads 
to  an  unending  Increase  in  available  precision  for  a  given  measurement.  While 
this  assumption  may  be  valid.  It  need  not  he  valid.  The  causes-of-error  data, 
for  the  assumption  to  be  truly  valid,  muatHse  parts  of  near  infinite  sets; 
all  must  be  essentallly  equally  likely;  and  the  contributions  of  each  must  be 
roughly  equal  for  a  valid  Increase  la  precision  to  be  possibly  obtainable. 

(Otner  conditions  may  hava  to  be  mat  as  well.)  Otherwise,  there  Is  an  optimum 
number  N  of  data  which  will  yield  as  much  precision  as  can  be  achieved,  and 
use  of  more  data  will  only  yield  an  apparent  ifl.pi ovement.  (The  behavior  Is 
somewhat  similar  to  that  of  an  asymptotic  function.)  Whst  this  mssns,  of  course, 
Is  that  the  results  achieved  mathematically  no  longer  represent  reality  since 
the  bias  In  the  original  data  are  obscuring  the  true  values. 

Unfortunately,  it  is  seldom  possible  to  determine  what  the  optimum  value 
of  N  to  be  selected  Is.  It  Is  a  function  of  all  the  parameters  which  could 
not  be  processed  In  a  deterministic  fashion.  The  variations  In  probability 
of  occurrence  of  the  different  causes  of  error,  and  the  variations  In  magnitude 
of  errors  all  contribute  to  this  Indeterminacy,  and  lead  to  the  limitation  on 
the  available  amount  of  improvement  of  precision. 

Statistical  weighting  and  evaluation  1s  applied  whenever  the  scatter  on 
data  points  Indicates  that  random  processes  are  disturbing  an  experiment. 

Where  the  range  of  data  variation  Is  small,  and  there  do  not  appear  to  be 
data  points  tdilch  fall  significantly  away  from  the  normal  grouping,  this 
procedure  probably  is  acceptable.  When  many  of  the  data  points  do  group  well, 
but  there  are  some  falling  substantially  awsy  from  the  remaining  group  lor 
no  readily  apparent  reason,  use  of  caution  and  a  careful  review  of  the 
experiment  both  are  essential  to  make  certain  that  unsuspected  factors  are 
not  disturbing  the  experimental  measurements. 
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The  e-me  theory  which  i>  used  to  juetify  the  discarding  of  en  epperently 
discordant  detua  actually  should  warn  the  user  that  there  la  something 
needing  more  investigation,  or  the  point  would  have  been  compatible  with 
the  rest  of  ehe  set.  Typically,  the  existence  of  auch  data  is  a  warning 
that  there  stay  be  dstarmlnlatlc  sources  of  error,  ones  of  much  larger  in- 
fluence  than  the  reat  and  occurring  store  frequently  than  the  rest,  still 
to  be  separated  and  studied.  These  data  cannot  be  Ignored  with  impunity! 

THEORY  OP  FURRY  SETS.  Many  of  the  measurements  which  one  must  make  In 
the  development  or  utilisation  of  an  experiment  are  of  dubious  value  at  least 
as  much  or  more  because  of  measurement  problems  than  becauae  of  random  errors. 
When  this  is  true,  it  Is  often  possible  to  develop  an  improved  understanding 
of  the  measurement  problem  and  its  nature  through  consideration  in  terms  of 
fussy  ssts.  Whereas  statisticsl  approaches  are  baaed  on  the  idea  chat  tha 
number  is  precise,  but  cannot  be  measured  precisely,  the  theory  of  fussy  sets 
implies  that  ths  number  is  only  determinable  to  a  limited  precision  because 
it  is  ill-defined  by  nature.  In  this  case  there  is  s  priori  a  limit  to  the 
amount  of  Improvement  which  can  be  obtained  by  use  of  statistical  methods, 

(In  shott,  s  measurement  problem  la  by  nature  probably  partly  statistical 
and  partly  "fussy-set".) 

It  has  been  known  for  at  least  fifty  years  chat  the  making  of  a  measure¬ 
ment  can  disturb  the  value  of  the  parameter  being  measured,  and  that  there 
is  In  feet  a  minimum  aeaaurable  element  as  a  consequence.  This  idea  is  basic 
to  the  development  of  quantum  physics  of  the  structure  of  matter,  but  it  has 
a  much  wider  range  of  applicability.  It  becomes  clearly  evident  that  this 
la  crus  when  the  measurement  of  energy  in  an  electromagnetic  wave  Is  attempted. 
At  high  energy  levels,  the  measuring  system  remove*  a  negligible  part  of  the 
incident  energy,  but  as  ths  incident  level  of  energy  is  decreased,  what  first 
appeared  to  be  a  wave  motion  begins  to  look  more  like  a  stream  of  particles 
(photons).  This  transition  cakes  place  as  the  seneor  requires  a  major  part 
of  the  Incident  energy  to  activate  the  measurement.  In  short,  the  process  of 
making  a  measurement  can  make  a  "fusty  set"  appear  even  fuzzier! 

INFORMATION  ENGINEERING  LIMITATIONS.  It  is  almost  axiomatic  that  experi- 
menta  tar  measurement  of  parameters  are  only  as  valid  as  their  mathematical 
and  physical  foundations  are  sound.  Since  In  real-life  experiments,  a  variety 
of  kinds  of  parameters  and  data  must  be  measured  in  a  meaningful  way,  it  is 
desirable  to  review  some  of  the  more  important  mathematical  and  physical 
criteria  which  must  be  taken  Into  account,  and  it  is  also  Important  that  the 
criteria  of  information  engineering  be  taken  into  account.  The  discipline  of 
information  engineering,  one  largely  ignored  In  at  least  some  technological 
fields,  iu  fundamental  to  proper  application  of  information  obtained  by 
experiment,  and  for  that  reason  will  be  considered  first. 
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The  Information  engineering  of  parameter  measurements  made  m  an  experi¬ 
ment  consists  of  two  principal  steps,  the  first  being  the  separation  of 
the  Information  Into  basic  categories,  and  the  second  the  establishment 
of  the  "order  of  Importance"  of  each  of  the  kinds  of  information  Involved, 

The  separation  Into  categories  is  required  In  order  that  the  user  may  have 
a  better  understanding  of  the  environment  end  detail  behavior,  and  typically 
the  two  most  Important  categories  are  frequently  the  ones  defining  the  basic 
operating  climate  to  which  the  experiment  la  to  be  subjected  and  the  actual 
active  measurements  to  be  made  In  this  climate. 

In  a  problem  involving  active  electronic  devices,  the  environmental  data 
would  Include  such  parameters  as  applied  voltages  and  currents  and  absolute 
temperature,  and  the  active  measurements  would  Include  carefully  selected 
small-signal  characteristics  of  tha  devlcs.  In  s  chemical  reaction,  the 
environmental  spaclf Icatlons  might  include  euch  paramatere  as  pressure, 
temperature,  state  of  materials,  catalytic  agents,  moderators,  iso-parameters, 
applied  energy  fields,  etc,,  and  the  detail  behavior  might  involve  specif lcatlona 
In  terms  of  derivatives,  etc.  of  the  iao-contoure  specifying  the  reaction. 

Stated  In  electrical  terms,  It  is  Important  to  Know  equipotentials  and  field 
gradients  in  terms  of  flow  rate,  or  tne  dual  functions,  in  addition  to 
special  environaantal  restrictions. 

The  second  step  in  information  engineering  is  one  of  ordering  on  the 
basis  ot  importance  ot  the  parameters  to  the  application  at  hand.  This  Is 
sometimes  considered  in  terms  of  a  series  of  experiments  starting  with  e 
zero-order  experiment  followed  successively  by  first-order,  second-order, 
etc,,  in  which  each  successive  experiment  represents  a  refinement  on  tne 
previous  one.  In  another  sense,  of  course,  the  meaauremente  made  should  be 
organized  to  measure  and  weight  data  on  the  basis  of  their  influence  on  the 
application  at  hand.  The  problem  of  ordering  these  data  la  a  problem  In 
determining  the  parameters  leading  to  the  "steepest  descent"  ana  organizing 
in  terms  of  relative  "rates  of  descent". 

As  an  example  of  such  a  characterization,  one  can  examine  the  typical 
plate  characteristic  curves  of  s  pentode  electron  tube  ae  compared  to  the 
corresponding  screen  characteristic  curves.  In  fact.  If  the  slopes  of 
plate  current  vs.  the  available  control  voltages,  plate,  grid  one,  grid  two, 
and  grid  three  are  determined,  the  respective  values  of  tne  transconductance 
order  as  follows: 

®ml  ”  *m2  >  «»3  >  8p 

with  many  tubes.  (The  order 

are,  the  reason  la  a  greatly 
suppressor  grid.  Typically, 
under  normal  use  conditions, 


or 


g  and  g  ,  may  be  reversed,  but  when  they 

P  ID.) 


reduced  effectiveness  of  control  due  to  the 
the  value  of  g  may  be  taken  to  approach  zero 
and  the  value  Sf  may  also  be  negligible. 
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The  values  of 


and  ,re  ALWAYS  important. 


Order  of  importance  has  one  very  unusual  aspect,  in  that  It  can  be  used 
to  show  that  a  email-signal  paraaeter  may  be  aore  inportent  than  an  environ- 
aental  paraaeter.  For  example,  with  a  typical  pentode  tube  type,  measure- 
aants  of  the  characteristics  will  frequently  show  that  a  wide  discrepancy 
may  exist  between  the  seta  of  static  curves  for  different  samples  of  the  aaae 
type  tube,  but  i  small-signal  paraaeter  may  be  specifiable  In  terns  which 
will  apply  with  ALL  of  the  samples.  Because  of  just  such  a  condition,  It 
la  possible  to  specify  pentode  conductances  In  terms  of  an  equation  of  the 
form 


<2> 

where  the  particular  chi  specif las  an  efficiency  factor  for  a  given  element 
of  the  tube,  the  Fermi  parameter,  A ,  having  the  value  (q/kT)  Is  familiar 
from  solid-state  physics,  and  the  current  I.  is  the  plate  current.  Equation 
two  may  also  be  used  for  determination  of  plate  conductance  by  the  appro¬ 
priate  substitutions. 

It  should  be  noted  that  Eq.  2  explains  why  so  much  design  of  tube  cir¬ 
cuits  has  been  based  on  stabilisation  of  output  current. 

Ocher  factors  in  addition  to  order  of  Importance  must  be  considered  in 
properly  organizing  the  Information  to  be  determined  by  experiment.  It  is 
Important  to  determine  the  stability  of  the  measurements  In  terms  of  possible 
measuring  parameters,  and  to  provide  data  In  terms  of  the  most  dependable 
configuration  of  parameters. 

For  example,  historically,  values  for  transconductance  for  tubes  have 
been  measured  as  a  function  of  place  current  and  voltage,  and  of  grid  and 
plate  voltages.  Examination  of  the  stability  considerations  Involved  (par¬ 
ticularly  In  terms  of  Eq.  2)  has  shown  that  the  proper  selection  of  charac¬ 
terising  variables  must  lead  to  use  of  plate  current  and  either  the  plate 
voltage  or  the  screen  voltage  (for  the  multigrid  tube),  and  the  experiment 
should  be  designed  for  measurement  of  transconductance  In  terms  of  them. 


MATHB1ATICAL  LIMITATIONS.  Tolerances  and  precision  are  normally  ex- 
preaaed  in  mathematical  terms,  and  they  are  of  particular  concern  in  the 
design  of  experiments.  They  often  enter  In  unexpected  ways  Into  experiments 
In  conjunction  with  differencing  errors  in  particular.  For  reasons  of 
convenience,  the  examples  to  follow  are  primarily  drawn  from  the  physics  of 
solid-state  active  devices  like  transistors. 
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On*  of  th*  most  Important  mathematical  limitation*  which  must  b*  recog¬ 
nised  and  C«k*n  Into  account  ia  th*  principle  of  small  dlf ferencea.  This 
principle  atataa  that  uaa  of  relations  dependent  on  email  difference*  of 
large  numbers  is  to  be  avoided  to  the  greatest  extent  possible.  Since  in  a 
sense  a  derivative  is  a  ratio  of  two  "small  differences".  It  is  Immediately 
clear  that  great  caution  la  mandated  In  th*  development  of  email-signal  data 
to  be  used  with  nonlinear  devices. 

Examination  of  the  nature  of  the  beta  for  a  translator  clearly  shove  the 
charactsrl  of  the  problem.  By  definition,  beta  can  be  given  in  terma  of  the 
equation* 

6*3  1/9  I.  (3) 

C  D 

where  the  base  current  Is  further  defined  by  the  equation* 

I.  +  I  +1  •  0  identically  (4) 

be® 

Since  for  any  good  transistor,  the  value  of  I.  is  small  compared  with 
either  I  or  I  ,it  le  Immediately  evident  that  beta  Is  defined  in  terma  of 
a  derivative  tfken  with  respect  to  a  small  difference  of  large  numbers! 

It  can  be  shown  that  the  mu  for  a  trlode  tube  is  defined  by  a  similarly 
uncontrollable  derivative  with  respect  to  an  unstable  variable,  the  grid 
voltage. 

An  examination  of  the  problem  of  coordinating  small-signal  and  static, 
of  environmental,  operating  conditions  shows  that  th*  sole  function  played 
by  the  environment  le  to  assure  that  th*  required  point-by-point  small-signal 
operating  conditions  can  be  developed,  this  swans  that  the  environment 
should  ba  stabilised  and  controlled  by  the  same  parameters  or  variables  that 
control  the  small-signal  behavior,  and  the  small-signal  behavior  should 
also  be  specified  in  terms  of  the  sane  variables.  Otherwise,  the  measure¬ 
ments  obtained  from  the  experiment  will  be  of  less  than  optimum  uttfulntas 
to  the  worker. 

Order-of-lmportance  of  stability  considerations  can  in  fact  be  rephrased 
In  terma  of  the  "small-difference"  limitation.  In  particular,  the  amall- 
algnal  parameters  which  may  be  of  use  with  any  nonlinear  device  should  ba 
examined  ona-by-one  to  determine  which  ones  can  ba  expressed  in  the  most 
stable  form  in  terms  of  physical  parameters  and  important  device  variables. 

A  tabulation  of  input  and  forward  lmmltcance  properties  for  the  bipolar 
transistor  then  might  take  the  form: 
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Parameter 

My  II 

"H** 

"g" 

Input 

A.  At 

ft  AI) 

ft  AI)' 

Forward 

Xlc 

ftl)_i 

ft  I>“ 

It  immediately  becomes  clear  that  All.  input  lnsslttances  depend  on 
A  I*  AND  both  tha  h^  and  a.  parameters  are  also  dependant  on  the  same A  I. 

Only  the  y^  la  not.  In  other  words,  not  only  the  input  but  also  the  forward 
lonlttance  parameters  at  well  are  dependent  on  small  differences  of  large 
currents  with  solid-state  devices  whose  characteristics  are  expressed  in 
terms  of  the  K  and  Z  configurations.  It  is  possible  to  minimise  the  con¬ 
sequences  of  dependence  on  AI  in  the  input  lamlttance,  but  not  in  BOTH  the 
input  and  the  forward  lamittancea.  “ 

CONCLUSIONS.  It  is  evident  from  the  above  discussion  that  the  design  of 
an  experiment  whose  purpose  is  to  provide  meaningful  measurements  is  a  rela¬ 
tively  complex  problem.  The  experiment  for  making  the  measurements  must  be 
designed  to  yield  meaningful  measurement  which  can  effectively  describe  tha 
characteristics  of  the  object  being  measured,  and  it  must  separate  deter¬ 
ministic  factors  from  both  statistical  and  fussy-set  factors.  In  addition, 
the  parameters  measured  must  be  ones  having  both  mathematical  and  physical 
validity,  and  they  must  be  stated  in  the  most  deterministic  form  possible. 
Above  all,  the  basic  criteria  for  use  of  statistical  and  fussy-set 
methods  must  be  kept  clearly  in  mind. 
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ABSTRACT 

This  report  develops  a  statistical  procedure  for  selecting  a  proper 
set  of  independent  variables  in  a  linear  regression  model.  The  procedure 
is  a  backward  elimination  procedure  in  which  initially  a  large  model  is 
hypothesised  and  systematically  non-significant  variables  are  eliminated 
one  by  one. 


Two  different  situations  were  Investigated  concerning  sample  estimates 
of  the  error: (1)  pure  error,  and  (2)  using  lack-of-fit  as  an  error  esti¬ 
mate,  with  the  effects  on  the  testing  procedure  for  each  case. 
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A  BACKWARD  ELIMINATION  GENERAL  SIGNIFICANCE 
REGRESSION  MODEL 


Section  I.  INTRODUCTION 

During  weapon  system  studies  and  investigations, 
there  is  no  shortage  of  "output  information."  At  the 
completion  of  most  field  tests,  measurements  and  many 
forms  of  recorded  data  abound.  Likewise  a  large  volume 
of  output  data  is  generated  during  a  study  where  many 
"computer  runs"  are  made.  To  properly  analyze  this  output 
information,  analysis  techniques  should  be  developed  for 
extracting,  from  the  masses  of  test  results  and  output 
data,  the  main  features  of  the  relationships  implied  or 
hidden  in  the  data. 

Ir.  investigating  any  process  or  system  in  which 
variable  quantities  change,  it  will  be  beneficial  to 
examine  the  effects  that  some  variables  exert  (or  appear 
to  exert)  on  others.  There  may  be  a  functional  relation* 
ship  between  variables:  in  electricity  Ohm's  law  states 
that  voltage  (V)  in  a  circuit  is  equal  to  the  product  of 
the  resistance  (R)  of  the  wire  and  the  current  (I), 

V  ■  RI;  or  Boyle's  law  which  states  that  at  a  constant 
temperature  the  pressure  (P)  and  the  volume  (V)  of  a 
given  quantity  of  gas  arc  related  by  the  equation  PV  - 
constant.  A  functional  relationship  between  variables 
is  frequently  too  complicated  tu  grasp  or  to  describe  in 
simple  terms.  In  this  case,  we  may  wish  to  approximate 
this  functional  relationship  by  some  simple  mathematical 
function.  By  examining  such  an  approximating  function 
nay  be  able  to  learn  more  about  the  underlying  true 

:onship  between  important  variables.  Even  where 
no  sensible  physica1.  (functional)  relationship  exists 
between  variables,  we  may  desire  to  relate  them  by  some 
sort  of  mathematical  equation.  While  this  equation  may 
be  physically  meaningless,  it  may  nevertheless  be  worth¬ 
while  for  predicting  the  values  of  some  variables  from 
knowledge  of  other  variables. 

Frequently,  during  many  weapons  system  studies  and 
investigations,  the  use  of  large-scale  long-running 
(machine  time)  computer  simulations  may  be  necessary; 
and,  in  many  cases,  field  experimentation  tests  are 
mandatory.  Examples  of  output  information  from  these 
simulations  or  field  tests  might  include  factors  such 
as  the  following: 
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a.  Attrition  rate  of  a  postulated  enemy  force 
from  a  computerized  wargame  simulation 

b.  Miss  distances  of  missiles  fired  at  targets 
during  field  tests 

c.  Detection  ranges  of  an  acquisition  radar  for 
a  particular  type  jet  aircraft 

d.  Visual  detection  ranges  for  an  observer 
versus  a  low  flying,  high  speed  jet  aircraft 

In  many  cases,  for  each  value  of  these  factors,  there 
art  a  number  of  concomitant  variables  which  are  known  and, 
in  many  cases,  fixed.  In  the  case  cf  b  above,  fixed  and 
known  variables  associated  with  each  test  shot  could  be 
(1)  target  altitude,  (2)  range  of  the  target  at  intercept, 
(3)  target  speed,  and  (4)  target  type.  For  d  above,  the 
fixed  variables  could  be  (1)  target  altitude,  (2j  target 
speed,  (3)  unmask  range  of  the  target,  (4)  search  sector 
size  for  the  observer,  and  (5)  flight  profile  type. 

Frequently,  once  the  computer  simulations  are  run  or 
the  field  tests  are  over,  there  are  a  multitude  of  "what 
if"  questions.  Cons'der  example  d  above  and  suppose  that 
during  this  field  test  a  total  of  three  different  target 
altitudes  (60,  200,  and  1000  meters)  are  used,  as  well  as 
two  target  speeds  (400  and  600  knots).  Shortly  thereafter 
questions  may  arise  concerning  visual  detection  ranges 
against  a  jet  aircraft  flying  at  an  altitude  of  2500  meters 
and  a  speed  of  750  knots. 

The  desire  to  address  these  "what  if"  questions,  as 
well  as  to  learn  more  about  the  underlying  relationships 
between  impoitant  variables,  necessitates  the  development 
of  a  procedure  or  model  for  such  types  of  analysis.  This 
report  is  the  results  of  such  an  investigation  into  the 
development  of  a  "prediction  model"  which  will  be  a  useful 
analysis  tool  for  use  in  weapon  system  studies  and  investi¬ 
gations  . 


549 


Section  II.  BACKGROUND  AND  GENERAL  METHODOLOGY  DEVELOPMENT 


Consider  a  system  or  some  process  whereby  for  a  given 
set  of  independent  variables  a  response  (or  output)  is 
obtained;  this  response  will  generally  be  referred  to  as 
the  dependent  variable.  Suppose  that  for  a  set  S  of  t 
independent  variables,  denoted  by  S  s  {X4 ,\3 , . . . ,Xt> ,  it 

is  highly  desirable  to  determine  or  estimate  a  relationship 
for  eventually  predicting  the  dependent  variable.  Consider  a 
total  of  n  responses,  each  denoted  by  Y^,  and  the  appropriate 

independent  variables  associated  with  each  response,  denoted  by 
Xii*Xji» • • • »Xti^ *  f°r  *  *  1,2,  ...,n.  Suppose  it  is  desir¬ 
able  to  approximate  the  relationship  (unknown) 


8(Xj^*X  -,...  *  X  ^  ^  ) 


with  a  relationship  given  by  the  model 


yi  *  VM1i^2x2i*...-ekx|<i*E,i 


for  i  -  1,2 . .  where  the  B's  are  unknown  parr  .eters  and 

k  <  n-1.  The  e.  term  of  (2)  has  conventionally  become 
known  as  the  "efror"  term  in  the  model;  it  can  represent 
experimental  errors  (measurement  errors,  incorrectly  cali¬ 
brated  equipment,  leaks  in  the  system,  etc.),  model  mis- 
classif ication  errors,  etc.,  or  a  sum  of  all  errors  from 
several  sources.  Also  no :e  that  the  parameters  are  all 
linear,  i.e.,  no  parameters  such  as  B^2,  Bj 3 .  etc.,  which 

implies  a  "linear  model."  Tne  basic  assumptions  are  made 
for  the  model  given  by  (2)  that 

a.  is  a  random  variable  with  mean  zero  and 
variance  cl  (unknown) 

b.  and  e ^  are  uncorrelated,  i^j,  so  that 

Covariance  (e^,e^)  *  0, 
and  for  hypothesis  testing  purposes  later  on 

c.  is  a  normally  distributed  random  variable 
with  mean  zero  and  variance  c’  (unknown). 
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The  system  of  observations  (responses)  given  by  (2) 
ca;i  be  written  in  vector  form  as 

Y  -  XJS  ♦  e  (3) 


where 

Y  is  an  (nxl)  vector  of  responses, 

X  is  an  (nxp)  matrix  of  known  fixed  quantities, 

£  is  a  (p*l)  vector  of  unknown  parameters, 
e  is  an  (nxl)  vector  of  errors, 
where  p  ■  k+1. 

At  this  stage  we  assume  that  the  model  given  by  (2)  is 
true.  However,  at  a  later  stage  we  must  test  to  determine 
if  indeed  it  is  true.  Draper  and  Smith  [3]  points  out  that 
we  are  "considering  or  tentatively  entertaining  our  model." 

Our  goal  is  not  only  to  obtain  estimates  for  the  parameters 
of  3,  but  also  to  determine  which  independent  variables  are 
contributing  significantly  to  the  response  of  our  mathe¬ 
matical  model.  Variables  that  are  non-significant  can  be 
left  out  of  the  model  for  prediction  purposes,  and  usually 
they  should  be  deleted. 

1 .  Development  of  a  Prediction  Equation 

References  [3]  and  [4]  show  a  generally  known  fact  that 
based  on  the  previous  assumptions  concerning  the  in  our 

model,  the  least  squares  and  maximum  likelihood  estimates 
for  £  are  given  by 

8  -  (X'X)*  lX' Y  (4) 

where  6_  is  an  unbiased  estimator,  i.e.,  E(§)  -  8,  for  8, 
and  where  X  and  Y  are  defined  in  equation  73) .  The  ('7  and 
(•*)  superscripts  on  the  matrices  are  the  standard  notations 
denoting  "transpose"  and  inverse,  respectively.  Since  8  is 
a  random  vector,  each  element  being  a  random  variable,  Tt 
has  an  associated  variance-covariance  matrix,  denoted  by 
Var(B),  where 

Var  (8)  =  a2  (X'X)' 1 . 
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Based  on  ouj  previous  assumptions  concerning  our  model 
it  follows  that  £  is  a  normally  distributed  random  vector 
with  mean  B  and  variance-covariance  matrix  a*  (X'X)*1; 
notationally  this  property  is  written  as 

N(B,oa(X'X)*‘).  (S) 

The  variances  of  the  elements  in  B  are  equal  to  the 
product  of  ca  and  the  diagonal  element*  of  (X'X)*1,  and  the 
covariances  of  the  elements  in  6  are  equal  to  the  product 
of  o*  and  the  off  diagonal  elements  of  (X'X)*1.  Notationally 
we  have 

Var  (Bj)  -  olCii 
Cov  (fij.ij)  -  °SCij 

where  represents  the  i,jth  element  of  (X'X)"1. 

One  statistical  technique  that  could  be  used  to 
determine  which  variables  in  (2)  are  significant  would  be 
to  test  the  hypothesis 

H„ :  Bj  ■  0 

H,  :  Bi  t  0 

for  i  =  1,2 . k.  For  prediction  purposes,  a  variable  that 

is  highly  non-significant  (fail  to  reject  H  )  could  be  left 
out  of  our  model;  variables  that  are  significant  (reject  H0) 
would  be  retained.  Using  this  type  elimination  procedure, 
one  could  develop  a  prediction  model  as  depicted  by  (2). 

Since  the  Bi,  for  i*l,2,...,k,  might  not  be  independent, 
the  results  of  (6)  could  very  well  be  dependent  on  each 
other.  This  could  lead  to  the  conclusion  that  a  particular 
variable  is  significant  when,  in  fact,  it  only  appears  that 
way  because  it  is  highly  correlated  with  some  other  impor¬ 
tant  variable.  We  need  some  way  of  assessing  how  much 
"good”  a  variable  is  doing  over  and  above  the  other  vari¬ 
ables.  Namely,  we  need  significance  tests  of  the  type 
given  by  (6)  where,  in  fact,  the  test  in  some  fashion  adjusts 
the  contribution  for  a  particular  variable  taking  into  con¬ 
sideration  the  fact  that  it  is  correlated  with  others.  This 
can  be  accomplished  by  the  "General  Regression  Significance 
Test"  for  which  the  methodology  is  given  below. 
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2 .  General  Regression  Significance  Test 

The  first  step  is  to  fit  a  "full  model"  as  given 
by  (2)  and  to  determine  the  estimate  £  for  8  as  given  by 
(4).  The  model  with  the  sample  estimates  for  the  unknown 
parameters  B0 ,  S , , . .  .  ,  Sj.  has  the  form 

yi  ■  .  .+§jcXjc+€j  (7) 

where  is  the  estimate  of  §.,  for  i  =  0,1,2,. ..,k. 

Associated  with  each  fitted  regression  equation  is  the  so- 
called  sums  of  squares  due  to  regression  which  convention¬ 
ally  is  denoted  bv  SSres  or  R 1 6 0 , 8 t , . . .  .  8^) .  For  our 

purposes  the  latter  will  be  chosen  and  at  times  will  be 
referred  to  as  R(all),  i.e.,  the  regression  due  to  all  of 
the  variables.  In  essence  the  sums  of  squares  due  to 
regression  is  the  variation  in  the  responses  due  to  the 
variation  in  the  independent  variables.  The  R(60 ,Bl , . . .  ,Bk) 
can  be  shown  as 

R(80,8lf...,6k)  =  I'X'Y  =  Y'tX(X'X)* ‘X'] Y  (8) 


where  the  X  matrix  and  Y  vector  are  defined  in  (3). 
Associated  with  any  sums  of  squares  is  a  parameter  known 
as  the  degrees  of  freedom;  its  importance  will  be  discussed 
later.  For  R(B0 .Bj , . . .  ,8.  )  the  degrees  of  freedom  will  be 
denoted  by  v  where 

v  «  k  *  1 . 


Suppose  we  are  interested  in  testing  the  significance 
of  some  8 ^ ,  the  hypothesis  which  is  given  in  (6).  As 

stated  in  paragraph  1  above,  we  need  the  regression  due  to 
8^  adjusted  for  all  of  the  other  variables,  which  is  denoted 


by  R(Si/80 . Bj - 1 , S . ♦ j , . . . , Bk)  ,  where  0  <  i  <  k  ,  or 

RCf^/all  the  rest).  This  can  be  determined  from  the 
equation 


R(8^/all  the  rest) 


R(30  ,3 


(9) 


RO 


♦  i 


•ek> 
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where  R($# ,$j , . . . ,8k)  is  given  by  (8),  R(Bt,Bt,..., 

8^- 1 ,B^+i , . . . ,Bk)  is  the  sums  of  squares  due  to  regression 

in  a  "reduced  model"  with  the  variable  left  out,  and 

0  <  i  i  k.  The  degrees  of  freedom  for  RC&^/all  the  rest) 

is  unity.  Recall  from  Section  I  that  the  error  term  in 
our  model  is  a  random  variable  with  a  variance  of  o2 
which  is  unknown.  To  test  the  hypothesis  given  by  (6)  we 
must  have  an  estimate  of  the  unknown  variance  o2.  Given 
an  estimate  of  o*  (or  error  estimate)  we  can  form  a  test 
statistic,  whose  distribution  is  known  under  the  null 
hypothesis,  and  can  test  the  appropriate  hypothesis  as 
depicted  by  (6).  Two  different  situations  concerning 
estimates  of  the  error  will  be  discussed:  (1)  the  case 
where  there  is  an  estimate  of  "pure  error"  (Case  A) ,  and 
(2)  the  case  where  there  is  no  "pure  error"  estimate  in 
which  specific  assumptions  must  be  made  (Case  B) . 

a.  Estimating  the  Pure  Error  (Case  A) 

Recall  that  we  have  a  total  of  n  responses. 
Let  there  be  a  total  of  s  distinct  sets  of  independent 

variables,  s^  *  *Xii,X2i . Xti^*  an<*  suPPose  for  each 

set  there  are  ni  responses,  for  i  ■  l,2,...,s.  Thus  we 
have  ni  replications  per  set  of  independent  variables. 

This  can  be  written  in  vector  form  as 


Y 

— n*  t 


*  i 


m+nz*. .  .+n. 


n . 


I  X  I 


n 


2  x  1 


SX1 


(10) 
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where  nt«“n4t...«-ns-n.  With  replications,  as  we  have  in  this 

case,  an  unbiased  estimate  for  the  error  exists.  This 
estimate  is  said  to  represent  "pure  eTror";  because,  with 
the  independent  variables  identical  for  two  responses,  only 
the  random  variation  can  influence  the  results  and  provide 
differences  between  them.  Let  the  sums  of  squares  due  to 

?ure  error  be  denoted  by  SSPE.  The  computational  formula 
or  SSPE  is  not  given  here,  but  it  can  be  found  in  many 
textbooks.  Denote  the  degrees  of  freedom  associated  with 
SSPE  by  dFPE  which  is  given  by 

dFPE  -  (n, - 1)+ (na-l)+. . .♦ (ns*l) .  (11) 

We  can  now  find  the  mean  square  of  the  pure  error  (MSE)  by 
the  ratio 

MSE  •  SSPE/dFPE  (12) 

where  E(MSE)  ■  a1.  With  this  estimate  of  the  error  we  can 
now  test  the  hypothesis  given  by  (6)  for  some  B.  of  interest. 
We  form  the  test  ratio  1 

F.  -  R(Bi/all  the  rest)/MSE  (13) 

where  R(8i/all  the  rest)  is  given  by  (9)  and  MSE  by  (12) 

The  distribution  of  the  test  ratio  under  the  null  hypothesis 
is  an  "F"  random  variable  with  degrees  of  freedom  1  and  dFPE. 
Therefore,  based  on  the  value  of  F^  in  (13)  and  some  nominal 

significance  level,  a  decision  is  made  concerning  retaining 
Bj  in  our  prediction  model.  A  test  statistic  can  be  formu¬ 
lated  for  all  B^  using  the  equations  given  by  (9),  (12), 

and  (13)  with  separate  decisions  being  made  as  to  whether 
or  not  to  retain  each  B^  in  our  model  for  i  ■  l,2,...,k. 


As  in  the  previous  case,  we  have  a  total  of 
n  responses,  as  well  as  s  distinct  sets  of  independent 
variables.  In  this  case,  however,  we  have  no  replications, 
which  means  that  n^n,* . .  .«n  «1 .  In  this  case,  we  have  no 

estimate  of  puie  error  which  means  that  the  error  estimate 
must  come  from  some  other  source. 


i 

I 

i 
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Consider  the  model  proposed  by  (2)  with  its  (k«-l) 
parameters  30 . Bl , . . . , 6^.  Theoretically,  in  any  regression 

model  the  total  number  of  parameters  can  equal,  but  not 
exceed, the  number  of  responses  (observations).  If  the 
postulated  model  (2)  contains  less  than  the  maximum  allow¬ 
able  number  of  parameters  it  is  said  to  suffer  from  "lack- 
of-fit.”  The  total  sums  of  squares  associated  with  our  n 
responses  can  be  partitioned  in  such  a  way  as  to  provide 
a  lack-of-fit  sums  of  squares,  denoted  by  SSLF.  The  SSLF, 
in  the  case  where  there  are  no  replications,  can  he 
determined  from  the  equation 

SSLF  -  Y'Y  -  |'X'Y  (14) 

where  Y'Y  is  the  total  sums  of  squares  and 

B'X'Y  is  the  total  regression  sums  of  squares 
“  given  by  (8). 

The  SSLF  divided  by  its  corresponding  degrees  of  freedom, 
SSLF/dFLF,  will  be  denoted  by  MSLF  which  provides  us  an 
estimate  for  the  unknown  error  (a*).  The  dFLF  can  be 
found  from  the  relation 

dFLF  -  n-  (k*l) 

where  n  is  the  total  number  of  responses  and  (k+1)  is  the 
total  number  of  parameters  in  our  postulated  model. 

Given  an  error  estimate,  i.e.,  MSLF,  we  can  proceed 
exactly  as  we  did  in  Case  A.  That  is,  we  form  the  test 
ratios  given  by  (13)  for  each  B^  and  draw  conclusions 

concerning  each  postulated  in  our  model  for  i  ■  l,2,...,k. 

Here  we  use  MSLF  instead  of  MSE  in  the  test  ratios. 

It  will  be  mentioned  briefly  that  using  the  lack  of  fit 
for  an  error  estimate  has  one  major  shortcoming.  The  mean 
square  (MSLF)  for  the  lack  of  fit  estimates  the  error  (o2) 
only  in  the  case  where  our  postulated  model  (2)  is  true. 

If  it  is  untrue,  then  the  mean  square  for  the  lack  of  fit 
(MSLF)  contains  not  only  random  error  but,  also,  systematic 
error.  This  can  be  expressed  in  the  expected  value  of  MSLF 
by 


E(MSLF)  -  c2  +  k 
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where  k  is  some  positive  constant  whose  magnitude  is 
dependent  upon  how  "untrue"  the  model  is.  If  the  model 
is  untrue  we  have  no  estimate  of  error  and,  thus,  no  valid 
test  statistic.  This  fact,  demonstrates  the  need  for  rep¬ 
lications,  and  in  most  situations  (field  tests  and  computer 
simulations),  replications  are  a  planned  part  of  the  study. 

In  summary,  the  following  steps  were  taken  during  the 
development  of  the  prediction  model: 

(1)  Pit  a  "full  model"  as  given  by  (2) 

(2)  Obtain  the  regression  due  to  each  8^  adjusted 

for  all  of  the  other  variables,  for  i  •  l,2,...,k,  given  by 

(9). 


(3)  Obtain  an  estimate  for  the  unknown  variance 
(a2) ,  Case  A  or  Case  B 

(4)  Conduct  k  separate  test  of  hypothesis  as  to 
the  significance  of  each  &.  in  the  prediction  model,  for 

i  •  1,2,. ...k. 

Section  III.  SELECTING  THE  BEST  REGRESSION  EQUATION 

Recall  from  Section  II  that  a  linear  regression 
equation,  or  prediction  equation,  was  established  for  a 
particular  response  Y  in  terms  of  independent  variables 
X  ,X  ,  ...,X..  A  full  model  was  fit  with  a  resulting  re¬ 
gression  equation  given  by  (7).  Methods  were  developed  to 
determine  if  any  variable  in  the  full  model  could  be  left 
out  as  faT  as  prediction  purposes  are  concerned.  Initially 
one  might  be  inclined  to  select  a  regression  equation  with 
as  many  X's  (independent  variables)  as  possible  so  that 
reliable  fitted  or  predicted  values  could  be  determined. 
This  could  be  disadvantageous  from  a  practical  point  of 
view  in  that  excessive  costs  and  energy  might  be  required 
in  obtaining  information  on  a  large  number  of  X's.  Also, 
it  is  extremely  important  to  remember  that  each  predicted 
response  is  a  random  variable  itself  with  an  expected 
value  and  variance.  Suppose,  for  discussion  purposes, 
that  the  full  model  given  by  (7)  is  the  final  model  we 
select  after  going  through  the  procedures  discussed  \n 
Section  II.  The  variance  for  a  predicted  value  of  r.  given 
a  set  of  fixed  X's  is  given  by 

VarfYj)  -  VarfB^B.Xj*.  .  .♦BkXk  +  ei] 
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or 


Var(?i)-Var(0#)^Var(BlXl)4.  ..♦Var(BkXk)*2X  Cov^.B,) 

♦. • «*2Xk. iXkCov(®k-i »®k)+Var(Gt)  US) 

Note  that  as  the  X's  increase  in  the  model,  the  precision 
about  each  predicted  response  decreases,  i.e.,  the  variance 
increases.  Since  this  is  true,  it  would  not  be  desirable 
to  include  a  large  number  of  X's  in  the  model  when,  in  fact, 
a  smaller  number  might  be  sufficient.  The  inclusion  of  too 
many  variables  in  tne  model  has  become  known  as  "over- 
fitting"  the  model.  The  procedure  for  selecting  an  ample 
number  of  variables,  guarding  against  over- fitting ,  is 
called  "selecting  the  best  equation." 

1 .  Selecting  the  Best  Equation 

There  is  no  unique  statistical  procedure  for 
selecting  the  "best"  regression  equation;  personal  judgment 
and  an  understanding  of  the  system  being  modeled  are 
necessary  in  choosing  the  best  regression  equation.  A 
literature  search  provides  a  rather  large  list  of  procedures 
that  can  be  used  in  developing  a  regression  equation.  Un¬ 
fortunately,  they  do  not  necessarily  lead  to  the  same 
solution.  The  procedure  used  in  this  study  for  choosing 
the  best  equation  is  called  "the  backward  elimination 
procedure"  and  is  discussed  below. 

The  Backward  Elimination  Procedure 


Many  steps  and  procedures  have  been  dis¬ 
cussed  in  Section  I  and  II  that  are  applicable  in  this 
backward  elimination  procedure.  Before  this  procedure  is 
described,  one  principle  pertaining  to  the  partitioning  of 
the  total  sums  of  squares,  Y^Y,  be  presented.  The 

total  sums  of  squares  can  be  partitioned  into  two  major 
separate  components  called  the  regression  and  residual  sums 
of  squares.  This  can  be  expressed  in  the  equation 

Y'Y  -  B'X'Y+[Y'Y-!'X'Y]  (16) 

where 

Y'Y  is  the  total  sums  of  squares 

|/X'Y  is  the  total  regression  sums  of  squares  and 

Y'Y-B/X'Y  is  known  as  the  residual  sums  of  squares. 
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Equation  16  can  be  written  for  brevity  as 

SS.  .  ,  •  SS  ♦  SS  (17) 

total  reg  res  v  J 

where  SSreg  is  the  same  as  R(80 ,Sj , . . . ,8^)  in  (8).  Theo¬ 
retically,  the  SS_-.  can  be  broken  down  into  two  separate 

quantities  -»  the  sums  of  squares  due  to  "pure  error"  and 
the  sums  of  squares  due  to  "lack  of  fit,"  denoted  by  SSPE 
and  SSLF,  respectively. 

Thus  we  have  in  equation  form 

SSres  "  SSPE+SSLF  O8) 

Suppose  that  a  regression  model  is  being  "entertained" 
in  a  situation  in  which  there  are  true  replications  in  the 
data  and  further  that  the  model  has  some  lack  of  fit.  That 
is,  we  did  not  include  the  maximum  allowable  number  of 
parameters  in  our  model.  The  possibility  exists  that  this 
lack  of  fit  may  be  statistically  significant,  which  would 
indicate  that  our  model  should  be  "enlarged."  Thus  we  need 
a  test  to  determine  if  the  lack  of  fit  is  significant.  If 
it  is  insignificant  we  would  conclude  that  our  model  suffers 
from  no  lack  of  fit,  which  would  imply  that  the  model  is 
large  enough.  If  the  lack  of  fit  is  significant  we  would 
conclude  that  our  model  should  be  enlarged,  in  which  case 
we  would  entertain  a  new,  enlarged  model.  From  (18)  we  see 
that 


SSLF  -  SSres-SSPE  (19) 

where  SS_^„  is  given  in  (18)  and  SSPE  was  discussed  in 
res 

Section  II.  It  follows  that,  with  an  estimate  of  pure 
error  (equation  (12)),  a  test  ratio  can  be  found  to  test 
for  the  significance  of  the  lack  of  fit.  We  determine  the 
mean  square  for  the  lack  of  fit,  MSLF,  given  by 

MSLF  -  SSLF/dFLF  (20) 

where  SSLF  is  given  by  (19)  and  the  degrees  of  freedom  due 
to  lack  of  fit,  dFLF,  is  given  by 

dFLF  -  (n* (k*l) ) -dFPE.  (21) 
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Recall  that  dFPE  is  given  by  (11).  The  null  hypothesis  we 
wish  to  test  is 

H# :  Lack  of  fit  -  0  (22) 

H,:  Lack  of  fit  ^  0. 

he  form  the  test  statistic 

Flf  ■  MSLF/MSE  (23) 

which  under  the  null  hypothesis  is  an  F  random  variable  with 
degrees  of  freedom  dFLF  and  dFPE.  Since  this  is  a  one-sided 
upper- tailed  test,  a  critical  value,  Fe,  at  some  specified 

significance  level  is  found  from  the  appropriate  table  of 
critical  values  for  the  F  distribution.  If  F^p  >  F  ,  we 

would  conclude  that  the  lack  of  fit  is  significant;  other¬ 
wise  we  would  fail  to  reject  the  null  hypothesis  and  con¬ 
clude  that  there  is  no  lack  of  fit. 

It  must  be  noted  that  no  test  for  the  lack  of  fit  is 
possible  if  we  have  no  replications.  With  no  replications 
we  have  SSPE  =  0  and  from  (18)  we  see  that 

SSres  *  SSLF‘  (24) 

Recall  that  in  the  case  where  we  have  no  replications, 
the  mean  square  for  the  lack  of  fit  had  to  be  used  as  our 
estimate  for  the  unknown  variance  (a*).  Therefore,  if  we 
have  no  replications  it  follows  that  no  test  exists  to 
determine  the  significance  of  the  lack  of  fit. 

We  are  in  a  position  now  to  describe  the  backward 
elimination  procedure  developed  in  this  study.  The  basic 
steps  are  as  follows: 

(1)  Fit  a  "full  model"  as  given  by  (7)  that  you 
feel  is  purposely  too  large. 

(2)  Determine  the  regression  for  each  variable 
adjusted  for  all  other  variables  given  by  (9). 

(3)  Determine  the  test  ratio  for  the  lack  of  fit 
if  we  have  replications. 

(4)  Form  test  ratios  for  8-  as  given  by  (13), 

for  i  -  1 ,2, . . .  ,k.  s 
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(5)  Determine  whether  or  not  the  lack  of  fit  is 
significant.  If  it  is  significant  enlarge  the  model  and 
repeat  Steps  (1)  thru  (S)  again. 

(6)  The  lowest  F-test  value  from  Step  (4),  F,  , 
is  compared  with  a  preselected  F  critical  value  at  some 
nominal  significance  level. 

(7)  If  F^  <  F0,  remove  the  variable  which 

gave  rise  to  F^,  from  the  model, and  recompute  the  regression 

equation  in  the  remaining  variables  and  repeat  steps  (1) 
thru  (7). 


(8)  If  F,  >  F0,  accept  the  regression  equation 
as  calculated.  L 

The  analysis  of  variance  (ANOVA)  table  for  the  first 
time  through  Steps  (1)  thru  (7)  is  shown  in  Table  I.  Once 
Step  (8)  is  reached  the  backward  elimination  selection 
procedure  is  terminated  and  those  remaining  variables  not 
previously  eliminated  are  the  ones  in  the  final  regression 
model . 


Table  I.  ANOVA  Table  for  Elimination  Procedure 


Source 

Degrees  of  Freedom 

Sums  of  Squares 

F  Ratios 

Total  (uncorrected) 

n 

Y'Y 

Regression 

k+1 

£  X'Y 

R(B0/all  the  rest) 

1 

R(B,/all  the  rest) 

R(S,/all  the  rest) 

• 

1 

R(B,/all  the  rest) 

• 

RfB^/all  the  rest) 

1 

RCB^/all  the  rest) 

Fk 

Residual 

Y'Y-g'X'Y 

Lack  of  Fit 

Subtraction 

(Y'Y-jTX'Y)-SSPE 

flf 

Pure  Error 

(n  t - 1 ) ♦ . . . *(ns- 1) 

SSPE 

61 


I 
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The  backward  elimination  procedure  has  been  computerized 
in  order  for  the  selection  procedure  to  be  "automated."  The 
Appendix  contains  descriptions  of  the  computer  program  along 
with  the  input  format  for  the  data.  The  program  prints  out 
the  ANOVA  table  each  time  through  steps  (1)  thru  (7)  along 
with  a  calculated  quantity  known  as  the  square  of  the 
multiple  correlation  coefficient,  denoted  by  R*.  It  is 
calculated  from  the  equation 

R2  -  (25) 

T'Y  -  nY2 

where  §"X'Y  is  the  regression  sums  of  squares  after  the 
completion  of  step  (7)  each  time  through  the 
procedure , 

Y  is  merely  the  mean  of  all  the  responses,  and 

n  is  the  total  number  of  responses. 

R2  is  a  measure  of  the  usefulness  of  the  terms,  other  than  B#, 
in  the  model,  where  0  <  R2  <.  1.  Generally,  large  values  of 
R2 ,  R2  >  0.90,  are  associated  with  models  that  fit  the  data 
very  well.  R*  might  be  used  as  an  easily  obtainable  measure 
of  the  success  of  the  regression  equation  in  explaining  the 
variation  in  the  data.  One  must  remember,  however,  that  it 
can  be  made  large  merely  by  including  in  our  model  near  the 
maximum  allowable  terms,  the  maximum  being  the  number  of 
observations.  The  multiple  correlation  coefficient  is  cal¬ 
culated  and  displayed,  not  from  the  standpoint  as  a  measure 
of  success  of  our  model,  but  rather  from  the  statisticians 
viewpoint  of  seeing  how  little  R2  decreases  from  the  initial 
’’full  model"  to  the  final  selected  model.  This  will  be 
shown  in  the  following  example. 

2 .  Example 

A  rocket  engine’s  performance  is  affected  by  a 
number  of  environmental  factors  and  conditions.  One  measure 
of  normal  performance  that  can  be  selected  for  analysis 
purposes  is  chamber  pressure.  A  typical  set  of  such  data  it 
shown  in  Table  II,  which  we  will  use  to  develop  an  empirical 
prediction  model  for  chamber  pressure.  With  the  data  given 
in  TabJe  11  let  us  entertain  or  nypethesize  a  model  which 
includes  the  variables  Xl(X.,X  ,  and  X^,  as  well  as  all  of 
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Table  II.  Rocket  Engine  Data 


Chamber 

Pressure 

(Y) 

Temperature 
of  Cycle 
(X,) 

Vibration 

(X2) 

Drop  Shock 
(X,) 

Static  Fire 
(XJ 

1.4 

-75 

0 

0 

-65 

26.3 

1 7  S 

0 

0 

150 

26.  5 

0 

-75 

o 

150 

5.8 

0 

175 

o 

-65 

23.8 

0 

-75 

0 

150 

7.4 

0 

1 7  S 

0 

-65 

29.4 

0 

0 

-65 

ISO 

9.7 

0 

0 

165 

-65 

32.9 

0 

0 

0 

150 

26.4 

-75 

-75 

0 

150 

8.4 

175 

175 

0 

-65 

28.8 

0 

-75 

-65 

ISO 

li.8 

0 

175 

165 

-65 

28.4 

-75 

-75 

-65 

150 

11. S 

175 

175 

165 

-65 

26.5 

0 

-75 

0 

150 

5.8 

0 

175 

0 

-65 

1.3 

0 

0 

-65 

-65 

21.4 

0 

0 

165 

150 

0.4 

0 

-  7C 
•  +> 

-65 

-65 

22.9 

0 

175 

165 

150 

26.4 

0 

-75 

-65 

150 

11.4 

0 

175 

165 

-65 

3.7 

0 

0 

0 

-65 

the  first  order  cross-products  between  variables,  i.e., 

Xj ,Xa , . . . ,X,X% .  If  this  is  the  case,  our  model  will  take 

on  the  form 


Yi 


(26) 


*67X1X^68X2X3^9X?X4  +  81  0XJX^ei  , 


Where  the  8's  are  unknown  coefficients,  and  will  be  esti¬ 
mated  from  our  sample  of  24  responses.  The  X  matrix  for 
this  situation  (equation  2)  is  given  in  Table  III.  When  we 
look  at  our  model  along  with  the  vector  of  responses  and  the 
X  matrix  we  see  there  are  some  variables,  especially  cross- 
products,  that  probably  will  be  insignificant  in  our  model. 
When  you  do  not  know  what  variables  should  be  in  the  model, 
you  purposely  hypothesise  a  large  model  (perhaps  over-fit) 
and  then  let  the  backward  elimination  procedure  choose 
those  variables  that  should  be  eliminated  and  those  that 
should  be  retained. 

Tables  IV  thru  X  (AN'OVA  Tables  1  thru  7)  are  coDies  from 
computerized  model  printouts  of  the  backward  elimination 
procedure.  ANOVA  Table  1  shows  the  regression  for  each 
variable  in  the  full  model  adjusted  for  all  of  the  other 
variables.  The  lack  of  fit  is  tested  the  first  time  through 
and  is  found  to  be  insignificant;  thus,  we  conclude  that 
our  model  does  not  suffer  from  any  lack  of  fit.  We  see  that 
variable  X,X2  is  eliminated  as  being  insignificant. 

Note  that  the  square  of  the  multiple  correlation  coefficient 
is  0.984769.  This  table  also  shows  the  sample  vector  of  the 
unknown  parameters  with  Ss  eliminated. 

We  see  in  ANOVA  Table  6  that  variable  X8  ?  X2X3  is 
eliminated  and,  in  fact,  Xfl  is  the  last  variable  to  be  elim¬ 
inated  as  indicated  by  ANOVA  Table  7.  Since  no  variable  is 
eliminated  in  Table  7,  the  backward  elimination  procedure 
has  determined  that,  of  the  ten  original  independent  vari¬ 
ables  included  in  the  full  model,  only  variables  X2,XS,X4 

and  XjXi,  should  be  retained  in  our  prediction  model.  The 
sample  estimates  of  the  unknown  parameters  S0,B2,S3,84,  and 

8,0  are  given  in  this  order  in  ANOVA  Table  6.  The  final 

prediction  equation  (regression  model)  is  written  as 

Y.  -  10. 6376  +  0. 0183X2-0.0106X3+0.  11  5Xi)-0.0003X3Xit  (27) 
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Table  .'V.  ANOVA  Table  1 


Source  of 
Variation 

Degrees  of 
Freedom 

Sum  of 
Squares 

Mean 

Square 

F 

Ratio 

Total  (Uncorrected) 

24 

9308.4500 

Total  Regression 

11 

9267.1495 

Due  to  6  0/rest 

1 

995.1493 

995. 1493 

539.2139 

Due  to  0  1/rest 

1 

3.8214 

3.8214 

2.0706 

Due  to  0  2/rest 

1 

31.3958 

31.3958 

17.0116 

Due  to  0  3/rest 

1 

19.1695 

19.1695 

10.3868 

Due  to  0  4/rest 

1 

1763.5096 

1763. 5096 

955.5440 

Due  to  0  S/rest. 

1 

1.7787 

1 . 7787 

0 . 9638 

Due  to  0  6/rest 

1 

3.0182 

3.0182 

1.6354 

Due  to  0  7/rest 

1 

7.0241 

7.0241 

3.8059 

Due  to  8  8/rest 

1 

5.2367 

5.2367 

2.8374 

Due  to  6  9/rest 

1 

2.7710 

2.7710 

1.5014 

Due  to  B  10/rest 

1 

121.2713 

121.2713 

65.7099 

Residual 

13 

4’  3004 

Lack  of  Fit 

7 

30.2271 

4.3182 

Pure  Error 

6 

11.0733 

1 . 8456 

Percentage  Variation  Explained  by  Regression  -  98.4769 


Variable  Leaving  X$ 

Vector  of  Unknown  Parameters  with  Variable  X  Deleted 

10.9908  5 

0.0070 
0.0196 
0.0160 
0.1134 
-0.0001 
-0.0001 
-0.0001 
0.0001  566 

-0. C004 


-75  0  0  -65 


O  i/>  o 

UO  sO  uO 


O  O  tS) 
vO 


i/>  tS)  o 

rs 


o  o  o 


LO  o  o  ^ 
'O  m  U">  \Q 


t-O  o  c  o 

NO 


o  o  *-n  1^5 

r-  r- 


o  o  ^ 

r-* 


OLOomOLntnOLnoOLnu^ 
u^sOLD'OuOvC  sO  lO  xC  LO  t/5  nD  nO 


L0L0unLnoOL0L0i/>0^i/>Lno 
\©  'O  O  'O  O  O  nO  nO  sO  nO 


tnu^t/>tr>i/>tnoOi/^Lnint/>o 


i/iinoooooooow 

r- 


565 


Table  V.  ANOVA  Table  2 


Source  of 
Variation 

Degrees  of 
Freedom 

Sum  of 
Squares 

Mean 

Square 

F 

Ratio 

Total  (Uncorrected 

24 

9308.4500 

Total  Regression 

10 

9265.3708 

Due  to  6  0/rest 

1 

11S0.6872 

1150.6872 

623.4910 

Due  to  6  1/rest 

1 

2.6233 

2.6233 

1.4214 

Due  to  S  2/rest 

1 

31.6996 

31. 6996 

17.1762 

Due  to  S  3/rest 

1 

20.0527 

20.0527 

10.8654 

Due  to  S  4/rest 

1 

1803.7715 

1803.7715 

977.3596 

Due  to  8  6/rest 

1 

4.9871 

4.9871 

2.7022 

Due  to  8  7/rest 

1 

8.7484 

8. 7484 

4.7402 

Due  to  8  8/rest 

1 

4.6604 

4.6604 

2.S252 

Due  to  6  9/rest 

1 

2.5714 

2.5714 

1.3933 

Due  to  8  10/rest 

1 

121.2032 

121.2032 

65.6730 

Residual 

14 

43.0791 

Lack  of  Fit 

8 

32.0058 

4.0007 

2.1678 

Pure  Error 

6 

11.0733 

1.8456 

Percentage  Variation  Explained  by  Regression  -  98.4113 


Variable  Leaving  X9 

Vector  of  unknown  Parameters  with  Variable  X9  Deleted 

10.7164 
0.0068 
0.0183 
.  .0155 
0.1125 
-0.0001 
-0.0001 
-0.0001 
-0.0003 
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Table  VI.  ANOVA  Table  3 


Source  of 
Variation 

Degrees  of 
Freedom 

Sum  of 
Squares 

Mean 

Square 

F 

Ratio 

Total  (Uncorrected) 

24 

9308.4500 

Total  Regression 

9 

9262.7994 

Due  to  8  0/rest 

1 

1517.2513 

1517.2513 

822.1109 

Due  to  6  1/rest 

1 

2.4660 

2.4660 

1.3362 

Due  to  0  2/rest 

1 

29.2196 

29. 2196 

15.8324 

Due  to  B  3/rest 

1 

18.9586 

18.9586 

.10.2726 

Due  to  8  4/rest 

1 

1847.6881 

1847.6881 

1001.1555 

Due  to  6  6/rest 

1 

4.1765 

4.1765 

2.2630 

Due  to  8  7/rest 

1 

6.6966 

6.6966 

3.6285 

Due  to  8  8/rest 

1 

2.7748 

2.7748 

1 . 5035 

Due  to  8  10/rest 

1 

179.5070 

179.5070 

97.2645 

Residual 

15 

45.6506 

Lack  of  Fit 

9 

34.5772 

3.8419 

2.0817 

Pure  Erro*r 

6 

11.0733 

1.8456 

Percentage  Variation  Explained  by  Regression  -  98.3165 

Variable  leaving 

Vector  of  Unknown  Parameters  with  Variable  X  Deleted 

10.7134  1 

0.0206 
0.0154 
0.1132 
-0.0001 
-0.0001 
-0.0001 
-0.0003 
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Table  VII.  ANOVA  Table  4 


Source  of 
Variation 

Degrees  of 
Freedom 

Sum  of 
Squares 

Mean 

Square 

F 

Ratio 

Total  (Uncorrected) 

24 

9308. 4S00 

Total  Regression 

8 

9260.3334 

Due  to  8  0/rest 

1 

1S16.4824 

1S16.4824 

821.6943 

Due  to  8  2/rest 

1 

4S. 3953 

45.3653 

24.5971 

Due  to  8  3/rest 

1 

18.6350 

18.6350 

10.0972 

Due  to  8  4/rest 

1 

1928.5725 

1928.5725 

1044.99821 

Due  to  8  6/rest 

1 

2.0264 

2.0264 

1.0980 

Due  to  8  7/ rest 

1 

4.4023 

4.4023 

2.3854 

Due  to  8  8/rest 

1 

4.5745 

4.5745 

2.478-7 

Due  to  8  10/rest 

1 

182.0314 

182.0314 

98.6323 

Residual 

16 

48.1166 

Lack  of  Fit 

10 

37.0433 

3.7043 

2.0072 

Pure  Error 

6 

— —  ■  ■  ■  -  ■■  -  ■ 

11.0733 

1.8456 

Percentage  variation  Explained  by  Regression 
Variable  Leaving  Xs 


Vector  of  Unknown  Parameters  with  Variable  X  Deleted 

10 . 6867 
0.0200 
0.0137 
0.1129 
-0.0000 
-0.0001 
-0.0003 
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Table  VIII.  ANOVA  Table  5 


Degrees  of 
Freedom 


Source  of 
Variation 


Total  (Uncorrected) 
Total  Regression 
Due  to  8  0/rest 
Due  to  8  2/rest 
Due  to  8  3/rest 
Due  to  8  4/rest 
Due  to  8  7/rest 
Due  to  6  8/rest 
Due  to  8  10/rest 

Residual 

Lack  of  Fit 
Pure  Error 


Percentage  Variation  Explained  by  Regression 
Variable  Leaving  X? 


Sum  of 
Squares 

Mean 

Square 

F 

Ratio 

9308.4500 

9258; 3070 

1515.9869 

1515.9869 

821 . 42S8 

43.6365 

43.6365 

23.6441 

16.6087 

16.6087 

8.9993 

1932.7930 

1932.7930 

1047.2690 

2.9298 

2.9298 

1.5875 

3.6566 

3.6S66 

1.9813 

190.3714 

190.3714 

103.1513 

50.1430 

39.069b 

3.5518 

1.9245 

11.0733 

1 . 3456 

93.1508 


Vector  of  Unknown  Parameters  with  Variable  X7  Deleted 

10.7079 

0.0202 

0.0138 

0.1127 

-0.0001 

-0.0003 
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Table  IX.  ANOVA  Table  6 


Source  of 
Variation 

Degrees  of 
Freedom 

Sum  of 
Squares 

Mean 

Square 

F 

Ratio 

Total  (Uncorrected) 

24 

9308. 4500 

Total  Regression 

6 

9255.3771 

Due  to  6  0/rest 

1 

1525.1281 

1525.1281 

826.3789 

Due  to  6  2/rest 

1 

44.3778 

44. 3778 

24.0458 

Due  to  6  3/rest 

1 

16.9036 

16.9036 

9.1591 

Dje  to  6  4/rest 

1 

1929.9972 

1929.9972 

1045.7540 

Due  to  8  8/rest 

1 

3.5612 

3.5612 

1.9296 

Due  to  8  10/rest 

1 

359. 1669 

199.1669 

107.9170 

Residual 

18 

53.0728 

Lack  of  Fit 

12 

41.9995 

3.5000 

1.8964 

Pure  Error 

■  - 

6 

11.0733 

_ 

1 . 8456 

Percentage  Variation  Explained  by  Regression  -  98.0427 


Variable  Leaving  X# 


Vector  of  Unknown  Parameters  \vith  Variable  X  Deleted 

10.6376 

0.0183 

0.0106 

0.1115 

-0.0003 
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Table  X.  ANOVA  Table  ? 


Source  of 
Variation 

Degrees  of 
Freedom 

Sum  of 
Squares 

Mean 

Square 

F 

Ratio 

Total  (Uncorrected) 

24 

9308.4500 

Total  Regression 

S 

9251.8159 

Due  to  6  0/rest 

1 

1533.4719 

1533.4719 

830. 8999 

Due  to  8  2/rest 

1 

40.9456 

40.9456 

22.1861 

Due  to  8  3/rest 

1 

13. 3751 

1  3 . 3  7  5 1 

7.2472 

Due  to  6  4/rest 

1 

2011.4026 

2011.4026 

1089.8629 

Due  to  B  10/rest 

1 

197.6084 

197.6084 

107.0726 

Residual 

19 

56.6340 

Lack  of  Fit 

13 

4  5 . 5b0b 

3.5047 

1 . 8990 

Pure  Error 

6 

11.0733 

- - - 

1.8456 

Percentage  Variation  Explained  by  Regression  -  97.9114 
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whore  the  estimate*  for  the  8's  are  given  in  ANOVA  Table  7. 
Observe  in  ANOVA  table  7  that  the  square  of  the  multiple 
correlation  coefficient  is  0.979114,  and  recall  from  ANOVA 
Table  1  that  this  coefficient  is  0.984769.  Thus,  we  see 
that  even  after  eliminating  six  variables,  there  is  only  a 
slight  reduction  in  percentage  of  variation  explained  by 
regression.  This  implies  that  our  final  equation  is  doing 
a  good  job  in  "fitting"  the  data. 

3.  Concluding  Remarks 

This  report  has  presented  the  backward  elimination 
procedure  developed  for  selecting  the  proper  variables  that 
should  be  included  in  a  linear  regression  model  for  predic¬ 
tion  purposes.  Several  assumptions  (explicit  and  implicit) 
were  made  that  were  necessary  for  the  development  of  the 
elimination  procedure.  If  these  assumptions  in  practice  are 
incorrect,  the  backward  elimination  procedure  or  any  general 
regression  procedure  will  be  invalid. 

Two  major  assumptions  were:  (1)  the  normality  assump¬ 
tion  on  our  error  term  (e^)  in  our  model  (explicit  assump¬ 
tion),  and  (2)  the  necessity  for  the  (X'X)  matrix  to  be  a 
non-singular  matrix  (implicit  assumption).  If  assumption 
(1)  is  incorrect,  we  have  no  valid  tests  for  our  variables 
in  the  ANOVA  table.  This  assumption  does  not  affect  the 
sample  estimate  for  the  unknown  parameters  B#,B, , . . . ,6^. 

The  topic  of  what  to  do  if  the  observed  responses  are 
not  normally  distributed  has  not  been  addressed  in  this 
study.  Needless  to  say,  it  is  an  important  topic;  however, 
the  literature  has  ample  examples  pertaining  to  various 
transformation  techniques  that  can  be  used  to  attain  the 
basic  requirements  of  normality  (References  [1],  [2],  [?), 
and  (5]). 

Assumption  (2)  means  in  practice  that  our  normal 
equations 


X'Xf  •  X'Y  (28) 

must  involve  as  many  independent  equations  as  there  are 
parameters  to  be  estimated.  If  the  (X'X)  matrix  is  singular, 
then  steps  must  be  taken  to  correct  this.  In  most  designed 
experiments  the  (X'X)  matrix  is  always  non-singular  and  thus 
no  problem  exists. 
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Another  topic  not  discussed  in  this  study  is  the 
treating  of  qualitative  or  deterministic  variables.  In 
the  model  proposed  in  this  study  all  of  the  independent 
variables  have  been  quantative.  over  some  continuous  range. 
Suppose  we  wish  to  include  variables  such  as  machine  number, 
factor  number,  vat  number,  missile  modification  number, 
etc.  in  our  model.  These  variables  need  to  he  quantified 
if  they  are  included  in  our  prediction  model.  Reference  [3] 
has  a  good  discussion  on  the  use  of  dummy  (qualitative) 
variables  in  multiple  regression.  Generally,  there  is  no 
problem  in  including  these  types  of  variables  in  a  regression 
model . 
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APPENDIX 


GENERAL  PROGRAM  DESCRIPTION 


Section  I.  INTRODUCTION 

The  Prediction  Model,  written  in  Fortran  IV,  uses 
double  precision  arithmetic  for  the  greatest  degree  of 
accaracy  possible.  The  program  was  written  for  the 
IBM  7094  and  runs  approximately  two  minutes  on  this 
machine . 

The  General  Flow  Diagrams  for  MAIN  and  FTEST  sub¬ 
routines  are  included  herein. 

The  model  has  the  capability  of  computing  answers 
for  any  number  of  cases  in  a  single  run,  provided  the 
deck  setup  is  as  described  in  Section  II,  Data  Inputs. 
A  complete  set  of  deck  cards  must  be  inserted  for  each 
case  desired. 

The  Fortran  IV  Source  Statements  are  given  in 
Section  III. 
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Section  II.  DATA  INPUTS 


-  The  data  inputs  for  the  Prediction  Model  are  lllus- 
trated  in  the  following  figures.  A  complete  data  set 
consists  of 

(1)  1  Title  Card 

■  ^<2)  1  Control  Parameters  Card 

(3)  1  Replications  Identification  Card 

(4)  1  set  Matrix  Entries  Card(s) 

(5)  1  set  Vector  Entries  Card(s) 

(.oj  1  F  Critical  Value  Card  for  Testing  Lack -of -Fit 

(7)  1  KF  Table  Identification  Card 

(8)  1  set  F  Table  Entries  Card(s) 

(9)  1  End  of  Deck  Indicator  Card 

The  cards  must  be  input  in  the  order  listed  above, 
and  a  complete  set  must  be  inserted  for  each  run  desired. 

The  input  card  formats  and  descriptions  arc  as 
follows : 
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SLRAi  PURPOSE  8  fiELO 


CONTROL  PARAMETERS  CARD 


■nnnnnnn^iigigmianianaaiaBBBPEaasgic 


gg)ii»nwn»a’mmDg]n»TPiB 


Enron  mrEiratf 


900000000 

111111111 


}  2  27?  2  2 2  2 


3  3|3  313  3  3  )  1 


4  4  4  4  1  4  4  4  4 


5 bib 5  J 5  b 5  5 


S  Sldlk  816  5  6 


1  11  1  ■  H 11  0 11111 II 11  II  11*1  II 11 II  ■■  lI'H'B  l'If  l'l  lrM  I'll'lrl'l  H  I  1 1111  H  I 


?  1 7171?  ?"?p  ;  7p  mi?'?  n  li  ?  TtTTT  ?  ?  W  ?  ?  ?  Min;  ?|?  ?  ?|? ;  mi?  >  ?im  ffr  ?  ?  p  ?  ?  ipT?  7p  m  ?  ?  ?  ?  ?  ?  ?  mi; 
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Field  Column 


Description 


NROW  -  number  of  rows  in  input  matrix; 
Maximum  value  =  30. 

NCOL  -  number  of  columns  in  input  matrix; 
Maximum  value  =  30. 

Not  used. 
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Column 


Description 

K  -  number  of  partitions  of  Y  vector. 

NREP  array  -  K  values,  each  of  which  repre¬ 
sents  the  number  of  replications  within 
each  partition  of  the  Y  vector. 
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Field  Column 


Description 


1 

2 

3 

4 

5 
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7 

8 
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10 


1-8 

9-16 

17-24 

25-32 

33-40 

41-48 

49-56 

57-64 

65-72 

73-80 


Entries  for  input  matrix  in  row  order. 

All  ten  fields  of  each  card  must  be  filled, 
with  the  exception  of  the  last  card,  which 
may  or  may  not  be  filled,  depending  on  the 
dimensions  of  the  matrix.  For  example,  a 
5x5  matrix  would  have  3  matrix  entries  cards. 
Card  1  would  contain  the  values  in  Rows  1 
and  2;  Card  2  would  contain  the  values  in 
Rows  3  and  4;  and  Card  3  would  contain  the 
values  in  Row  5  (with  the  last  5  fields 
blank) . 
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Components  of  input  vector;  all  ten 
fields  must  be  filled  with  the  exception 
of  the  last  card,  which  may  or  may  not 
be  filled,  depending  on  the  dimension  Ox 

the  vector. 
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KF  TABLE  IDENTIFICATION  CARD 


Field 

Column 

Description 

1 

1-2 

KF  -  Number  of  F  Table  entries  to  be  made 

2 

3-80 

Not  used. 
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END  OF  DECK  INDICATOR  CARD 


Field  Column 
1  1-6 

?.  7-80 


Description 

Indicates  procedure  program  should  follow: 
Value  =  EOJbbb  if  this  is  the  last  or 
only  data  set. 

Value  =  EORbbb  if  another  data  set 
follows . 

Not  used. 
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VARIATION  EXPLAINED  BY  REGRESSION  - 
UNKNOWN  PARAMETERS  WITH  VARIABL 


17  FunyAT<  1*1  .  10X,12Atl 
lb  FwFy  AT ( //  ) 

123  roR"AT  (  Ml  ,  10X,2AH.NC()L  GT  5,  CHECK  FORMATS  ) 

111  ECM  AT  I  HI ,  59X,5rtANuVA  ) 

Ini  FOR'  AT ( // / / « 2  9  X  , 7AHbnuRCE  UF  UkGREES  OF  SJM  OF 

1  mean  r  i 

ia2  Fj.’v  AT  (  29x  ,  7ohVAKi  AT  IV,  FREEDOM  SQUARES 

2  SjJAnE  -ATM  ) 

1  s 1  FoRv AT ( / , 2  AX - bAli - 


' . . . . . > 

1 o-  FuRV AT ( // ,2  AX, 19HTUTAL  (  UNC uRR cL T L 0 /, 5X , I  2 . 1 1 X , F 12 . A » 

IBS  FJMATI  /  //  »  2  AX ,  IjMITuTAL  REGaCSSICN  ,6X  ,  I  2  «  1 IX  ,  F  12 . 4  ) 

IW6  F  .  a  K  a  t  I  /  /  >  2  7X  •  EHOU  E  TO  3 , 1  2 , l AH/RE ST  1  ,  1  3  X  ,  F  ID  .  A  ,  5X  ,  E  10  .  A,  5 

sx.HC.A) 

1  37  FUFaAT  (  /  /  /  *  2  A  X  ,  b  H«  CS  1 0  'J  A  L  «  l  6  X  ,  I  2  ,  1  3  X  t  F  1  0  •  A  I 

1HH  r  Jr  y  *«  T  (//  >  2  7X  ,  1 1  ML  AC  K  UF  FI  T  ,  1  0  X  ,  I  2 , 1  3  X  ,  F  1  0 . 4 , 3  x  .  F  1 0 . 4 , 5  X ,  F  1  0 . 4  ) 
139  F'jtvATI// ,27X,llHPUr  E  ERkJR  ,  l  0  X  ,  1 2  *  13  X  ,  F  l  0.  A,  3  X  ,  F  1 0  .  A  I 
Ml  Fj.-ATI  iHl,  29X,37MHETA  HAT  VARIANCE  -  CUVARIANCE  MATRIX  ) 
nC  1  FoR“lT I//.A2X ,F10. A) 

4  C  2  FM-AT{//,3SX,2<FlC.4,5xn 
a  D  3  Fdav AT { // ,2 TX ,31  FI  0. A, 5X1) 

A  r  a  FjV- AT  I // ,! 0X,a(F1C.A,5XI) 

ADS  F  JRMAT  <//,laX,MHC.A,5X>) 

CAU  ELIM 

12; g:  on  23  I  *  1,2c 

i)  MCI  D  <  I  I  «  MALPHA!  I  ) 

c  Initialize 
ms  ■  1 


2 


Vi 


1 

■I 


I 

I 


Best  Available  Coo 

2  J: 


59n 


o  o  c 
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Fortran  IV  Source  Statements  (contd) 
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sc,'  SiUG  «  SSR  EG  ♦  CiT  X I  11  •  Till 

CwK^uTE  UGKtSSlJ‘1  DUE  TO  EACH  iiETAII) 


728 

730 


020 
5  70 


.jxljTc  UNC  CULU^M  rsCM  x  MATRIX 

S  S  7  =  •  0  ^ 

n j  72  0  1  *  l.-'XULl 
S\ t-».  <  I  «0'7 

x  XY'.I  I  »  i  0.00 
r,j  7  20  J  -  l.NCULl 
X  'M  1  t  J>  s  C.’JC 
CC:.T  t'JUE 

O'  73:  I  =  l.NRCM 
x  v.  *  I  l  )  =  0  •  0  C 
•«Cu.  I  -  VC0L  -  1 
j"  eo:  i  s 

X  ",  rt  I  It  -  C-.uO 

IFIlFLAG  .Nfc.  1»  OU  Tu  07s 

uCTC&'MNE  XNE »  FUR  oO 

Pj  SiO  I  '  1  » NH  Li  M 

00  ^25  J  *  l.fXULl 

x-,;»(  i  .J)  *  xi  I .  j*  n 

00  TO  720 

i.xi,;  *  i  ‘.o  i  ♦  i 

00  V  0  ■-  I  *  1 » NROW 


149 


180 


18  1 
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Fortran  IV  Source  Statements  (contd) 


rEoA 

NA  IN  -  sKN  sUUKC,  e  STATEMENT  -  IFNI  •> ) 

jj  :  j  «  l, iNiu 

6t:  XNEr.<  I  ,  J  )  «  Xtl ,J) 

JMlt  *  IND 1  *  1 
IHINDD.sQ.NCULJ  CO  TO  720 
T.J  7.'.  I  •  l.NROW 
JO  7C C  J  *  IND2.NCOL1 
7 :\t.< i  .j)  •  x 1 1 » j* 1 1 

CLk, jTt  NEk  BETA  VECTOR 
TRANSPOSE  XNEw 
72D  Oh  75C  I  «  lf.\RC« 

•J  7s:  J  *  l.NCGLl 
7sC  TxNl  h (  1,11  >  XNEWI I, J) 

'•‘ULtlPLY  XN Ck  9Y  ITS  TRANSPuSE 
Ji:  775  I  -  1 1  NCuLl 
•)  I  775  J  •  l.’.CGLl 
775  L  *  liNPOw 

775  *  TXNEkU.L)  *  XNE W I L  »  J I 

» i  T:  '  '•  t\f  INVERSE  C.  X.MN 

CaU  St  SO!'- 1  (  XuNf  NCCL  1  •  NCUL  1 » MS  »  RN1  *  U tR  *E  *HOK  K»  I HLI) ) 
■tULT  I  Pi.  Y  Y  VfcCTCR  BY  TXNbW 
00  7  0  1  ■  l.f.CuLl 
GO  7oC  J  «  1«\Vjw 

7b:  x.xvr  (M  «  XMYN(II  *  TXNeWU.J)  •  VC  J I 
''•JLUPLY  XMN  INVERSE  BY  X5YN 
DO  7b5  I  «  1«  f-CLLl 
01  7b 5  J  «  l.I.CGLl 

735  sNcw  I  I  )  «  HNcWlN  ♦  XMNII.J)  *  XMYN(J) 

IFINP4SS.Ew.0l  CO  TO  7VI 

in  *  a  ♦  i 

-a  ITE  (  6<7  1 1 i<OIC(  ILH) 

CO  TD  7  72 
7«1  «?.  ITEI  6f  7  J  IE 

7V2  *RIU(o,6)  (ttNEx  ( l  1 . 1  ■  1  .U.UL  1 1 

COMPUTE  DEGRESSION  FOR  NEW  X-S  ANO  05TA-S 
DO  rZC  I  «  l.NCOLl 
CO  50 C  J  ■  l.CRUW 
tCC  T*Vl I, JJ  ■  XNEWl J* I J 

oo  ?:?  j  ■  i.nrow 

JJ  SCO  I  -  X,  .NCULl 

SCC  XYRUJ  *  X  HR  ( J 1  ♦  uUE«  (  I  I  •  TXRU.JI 
00  K  DO  I  -  i.NROw 
: c:  ssr  ■  ssd  ♦  xmruj  •  ycii 
a  ■  ii  ♦  i 

IFLAC  ■  IFEAG  ♦  1 

ssai i  iu  «  sSbEC  -ssa 

IFllL.tC.NCOL  )  GO  TO  1 100 
GJ  TD  IOCCO 

ICO  IHMNO  .EC.  II  GO  TO  3C 

JETtRxlf.t  PURC  EkRO* 

CALCULATE  T  JOT  VALUES 
Lt  «  ! 

LU  ■  3 

DO  I3Dv  ‘X  «  1. K 
;i RP  ■  r.  .EPIKM  ♦  LU 
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Fortran  IV  Source  Statements  (Contd) 

-  £>',  iOOKte  STAT  E^tUT  -  l  F  NC  S »  - 


IS  i‘. 


.3  . 


m. 


LO  1  *.»** 

14-.;  |  *  LlfLU 

TO-:  1  *  T'.XM  ♦  V<  1  ) 

LL  *  -l  ♦  Nr*  E  fM  X  lO 
r.vM  i\wt 

VJU  b.UAt-fi  *oP  T  iJ  FUKt  t-  KMilK 
3 ..  l  b  .  C  I  *  I  ,I.-4U« 

-  aJ'JY  ♦  Y(  I 
•i  '  .  5  >  v  I  *  l .  K 
A  “  »  '*  t  !  >  >  'irt.HU  | 

I  •  '.o  '  1  *  1  |X 

j.l.  T  =  WIT  ♦  (  T{  I  )  >•*•?»  /X«o  P  <  n 

>.>•:  =  J J Y  -  SUIT 

1.  ;)Tt  OivWES  OH  reicKv  F (ik  S SH»E 
17'.  uv  *  1 » k 

=  !,  .FPL  ♦  Mfr'UK  I  -  1 
<t  -■<  StJA'- L  t1 
l>v;  s  i  If  ►’L 
•  ')i  -  "iSi*  t  /Xuf  >*  t 


f'L  V 

c....: 

r  , 


TEST 

\  4.3 

I  v  SSI-  i 

, .  I 


x£3&S&sl 


,  be*1 


A\o  SSLP 

4  i'nA'J* 


19-  - 


■  V  Y  1 V,  *  * 

YSl  M  ♦  Y(  I) *'2 

-  j  -  s 

YjJM  -iSAEO 

i>l? 

S  YcUa  -  SCtiL 

;*•  ( ■.  i 

u*c  4  ■  1  I  vivi  TO  3  3 

ijL>  * 

Sic  -  SSPc 

cal:>4. 

A  T  C  o'  . h  •  F  Is  TOT 

I  w  M  f 

*  IJFTE  -  U3FPS 

X  .  r  l  f 

*  l  0  F  L  F 

x 

=  SSlF/xDFtF 

FlF  * 

</Sl  f  /hr  St 

s.. 

:  i  *  i,r<sui 

:  ;  i  = 

3S-  l< I  l/KMSE 

s.  T  :« 

j  3 

3  3  <  jf 7 S 

*  l.irTC 

1  i  3  /  X  o'  r  T  c 

>  y  i  *• 

i  *  l«  •  *C LL 

34  r-  (  ;  |  = 

jS>*  till  /  ass-r 

ANC  LACK  OF  MT  -  PEFFOPP  F  TEST 


CAoCJiATt-  n.  SC 

1 J  .)  ,  i  i  *  i  t  N  P  C.  * 

3  i  Y  $  —  VS  ♦  Ydl 

v  3  =  *SJ*J 
< 

*  (Si'ES  -  YS/Xr.Owl  /  (  YS'IK  -  YS/XKOxI 
:.i.  *  ’  C  .0  *  kSw 
44  i  7  *  (  S  t  l oS I 

aaiti  i6,m  > 
a  *  i  7  - ( * «i2c) 

•»4  »T  t  (  H>«  l  L3  I 
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■A  AtH 


-  EFN 


tC<s  STATEMENT  -  1ENIS) 


**iu<6U6a>  \Pw'«.vsc« 

.,e,nei e>.i*»s>  ncll* ss*f u 

^  1,3  *!  y'yi\  in.ssfum.ssftun.Fm 

U?  K  H  U 

I-iuUavn  :oFPt,ss«*e.^fc 

\XCV  •<*  IT 6  ( t>*  13 1  ^SQ 

USTt)?»EU»l  >  60  Tu  *** 

•  S.  ;T£t  -.•!<.»  INOeXtNPASSl 

nFLAG  *  1 

SlKtillll  l.U>.l-l.KC0L» 

{FLAG  *  0 

Co  TO  31111 


,e.ITl  <«EU  HAT  VAKlAi.CE 

u;  a;*c<«*.rai 

i;!  iiiTiU,.-.ai'wii-"-j'i'NC"u 

GO  TO  120 

jtj  S .? 5f •Mai-WS « » . J . . ^.1. HC* » 

Gu  T'J  120 

K,3  '^ITUtsUuVuSa.JJ.J-l.NCQLl 

GU  TO  120 

Go  T 0  12P 

GO  TO  125 
Ui  IT  E  l  0*123 1 
12**  KcAo(5ei21  XIND 

IFUlf»0.cG.EUJl  60  Tu  333 
GO  TO  12C00 
33  2  3  STOP 


COVARIANCE  MATRIX 


Best  Available  Copi 


595 


r‘r>»’'  r  >  r>  r>  oor. 


Fortran  IV  Source  Statements  (contd) 


r  r  S  T  -  r.K.  Si'l-Cc  STAUMFNT  -  IFN(S) 


iof.  JT  I\F  Ml  ST 

CuW.  x(  3C  .  3M.fl  3‘  ),F  17.  11  t  23  I  ♦  •ijS'K  (V  1  ,  FI>,FCR2,MHD,KINU,NP6SS 
i  .  .CoL  .  *.*- J  ^  ,  I  Ti  *  I  iiF  >•(  ,tf  ,  (STOP,  IM'cXl  1C  » .  INt>I  C  (  20  » 

.  'S.'-'M.ImLM  Ml 
. ! ; :  f  Mi.-.  *‘i(jC,jOJ 

^iMSlu:.  »  ,r  ,F  Ti.'L  .1  L-  l  ,XN,F  L.l  LF  ,0,  WORK 
1  rt.O-f  T(  in*.  ,  ICX.t  •  >F  tlT  SIGUF1CANT,  R  F-e  *  AM  1  n£  "Hfc  RraKfcSSt 

t  '  «  *  ^  w  ••  II  ./»  ) 

t  •'.i-'f  *  i  ii: ,  loxo  mc..f.v.-iULifi  ms,  roiK  regression  has  oeen  fujno 

J  Vt.jAT'  I 

i  I,  -  'AT  (//////////,  £<>X  ,  if.rtVA-  IA,)tC  LfcAVIr.u  X.I2I 
-  c.ti,  «ishma»i  N  T  S.IFPI.'UM,  INpX  *  ,13) 

iC’.T  =  ! 

,R,V.  t  =  '.•'ASS  ♦  1 

i*  i .  I'.  /.v. .  n  ii.i  tp  s . 

Cm  •  F.  -v  Sli'.'.lF  ICA'.Cr  uF  LACK  CF  FIT 

li  C«PiSj,jT»l)  uu  Tii  1 1, 

IFIri  f  ...T  .FF>2)  GO  Tu  20 

1  *  J  i  V 

-C  .-ITEU.U 

1,*  Lli  - ■  I  I  VJZ 

JCh.-  ‘f.c  ShalLcST  F  i<AT|  j 

3  »  f  L  -  '  (  i  1 
IF  =  > 

s  ; :  1  =  3, 

I  f  III  :  ) ,3T , FL  >  jU  TC  2  : 

b  L  =  FID 

ip  =  i 

3 :  t  I'.iil 

t  .  .Ji  A  I  A  S  S  )  =  IF  ♦  GRASS  -  ?. 

1  F  A  S  j.  t  w  •  l  )  GO  Ti.  3  S 
•,R  =  .PASS  -  l 
)  1  j  I  =  »,N»* 

liil-wum  .lit.  l:.,.txe.RAbS)  »  I  CM  *  1C  NT  ♦  1 
3*  Cv/.TI'.Ur 

1.1,1  A I  GRASS  >  X  |.«:u>  C.,»ASS>  -  1CNT 

cv;  t.p  i  fl  «,  i  th  critical  valuc  frlh  ftabl 

35  C  J  '«*  i  l  > : 

1  *7  1  3  A  ,  I.C  L»l 

1  F  {  1  x(  VP  AS  S  I  .  '.F  .  1  .’.J  I C  (  I  1  I  GO  in  3? 

IV.  =  1 
...  T  3  ‘  *1 

>  I  b...  .1  i  ‘.111 

C  t L  e  T  L  IN)  Lx  F«,..m  iMUi.  APrAY 

- 1  ‘ . C i .  *  ..Col  —  i 

:-r  i  =  if.u.r.co 

.*3  i i :  i  1 1  =  i.-.cicc  i  *i  i 
IF  1,1  ir,*  1 1  Oi,  Tl1  a  , 

PC.  »  I'JFpr 
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Fortran  IV  Source  Statements  (contd) 


«ECT 

FTST 


Ef-N  SOURCE  STATEMENT  -  IFN(S) 


0-  T 0  7C 
60  INCA  «  10FTE 
73  Col.TINUE 

IFtrTAbU  IN3X  J.EU.  3. DO  )  TC  16. 

IF(rL.LT.FTABLUNDX)  »  Gu  TO  «: 

IMNPASS.Nfc.ll  GO  TC  65 
•« lTe( 6,2) 

E5  I STUP  •  1 
dC  CONTINUE 


OEttTs  COLUMN  FRCM  X  MATRIX 
[NO  -  IF  -  1 
00  V 3  I  ■  l.tiKO* 

00  TO  J  *  1 1 1  NO 
AUI,-J>  •  XU,J» 

.,CCL  *  KCwL  *  1 
Jj  ICC  1  *  1 « NBOv» 

0  J  1C?  J  *  l  F  ,  NC  Jt 
A, 4(1,  J  I  ■  X(I,J»1» 

O  j  110  l  *  1  *  KPilM 
DU  ill*  J  ■  ltNCJL 
X (  l.JI  ■  XNU.Jl 
IFUSTCP.E0.1I  GO  TO  U'O 
ITE (6,3)  INCEX(NPASSJ 
OJ  TO  120 


low  RAITE«6,«.I  INOX 
IST3P  •  l 
120  RETURN 
END 


Best  Available  Copy 


nor  oor»r»r>r>r>f'>or.or«r«r»r»r»r»r.  ononr.  r  r>ncr  «»r>««r-r»r»oc««»<tr>r'or>or*r>r»<-  r.  r»  <h  r  • 


Fortran  IV  Source  Statements  (contd) 


jSSJSb 


,F\  j .URLE  STATEMENT  -  1FNIS)  - 


iUu'C.Jl  1M  SESCMI  X.N,  M».bs,MM  .C.R.  fi,  WORK,  IHLD) 

CATEGORY 

•lilncMAT  l  CAL 

purpose 

SJIVf  SlMLLTANe'.US  CI.-JATI.JN'  OR  INVERT  MATRICES 
tic  3c H  I  P  T 1  c  N 

T.Uj  SJfSCUTtNC  >1  LI  soLVE  AN  I,  IT Y  N  SVSTEM  Cr  .>  I MUL  T  ANE  (klS 
EQUATIONS  kITh  AN  ARSIT'.ARY  NUMBER  JR  R  I  GmT  HAND  SIDES  OK 
INVENT  A  MATRIX  lit-  ,i^  If  >  tim  IN  Tut  PKtlCCSS.  TmL  MANX  UP 
TMt  MATk  I  x  AND  ITS  jITl-MINANT  Akt  EVALUATED.  THE  METHOD 
JSc.)  IS  ThAT  OF  GAUi  S-J'lRbAN  WITH  TuTAL  pivoting.  the 
MJUTINc  “AV  bC  RUN  IN  SlNGLC  CM  DjUBLE  PRECISION  ARITHMETIC, 

input 

1  *  EIkST  1..*.AT|.:n  O*-  INPUT  CUeFF.lCIENT  r.A  f  k  I  X  ,  X  I  1 , 1  ) 

AJ,*.**'I.Ti  J  .1 Y  Nc  RIGHT  HAND  SIDES.  ruk  MATRIX 
INVERSE  •  x  IS  HR  ST  LOCATION  (IF  THE  MATRIX  TO  RE 
INVERTED.  ;.l.  *11.1).  X  MUST  RE  Ul  McNS  lUMfct) 

T '.)  CM  . «  ,1  If.  TitE  CALLING  program  in  CITHER 

CASE. 

2  N  <Ov.3ck  OF  SIMULTANEOUS  EQUATIONS  Tu  BE  SOLVED, 

^  Ut-UeR  .>  MATRIX  TO  rit  INVERTED. 

2  NX  I  .*-  MIGHT  HAND  SIDES  FGu  SIMULTANEOUS 

LjUAIluN  S  I L  L‘  T I  C  N  *  ND  *  N  FOR  .MATRIX  INVERSE. 

A  *S  MS  *  •.  FiiR  SIMULTANEOUS  F  UOAT  I  )N  SOLUTION. 

«s  *  •  F.jf.  MATRIX  INVERSE. 

S  mm  Min  DIME  MIC;.,  Or  X  AS  DEHNEO  IN  CALLING  PKUGkAM . 

b  -jPk  V.OMKIND  AkMAY  L- 1  PENSIONED  AS  FOLLOWS  IN  CALLING 

PR.C&kAP  .  ..  o  jR»  I  .MN 1  *NO )  . 

7  I  ML  D  wOkKlNU  A-vAY  DIMENSIONED  AS  FOLLOWS  IN  CALLING 
(’ kOG1  av.  .  .  IhlOIMMI. 

OUTPjT 

;  x  xd.ii  thkjujh  xu.n  contain  first  solution 

VcCTi.k.  xll.21  Through  x  IN .  2  )  CONTAIN  SECOND 
SCLUTIIN  VrCT-jk,  ETC.  FOR  «atp|X  INVERSE,  THE 
tMKAY  X  C.'.TAJNS  the  inverse  matrix. 
i  0  GET  c“  M|  I, ANT  or  input  X. 

0  r  -AML  OF  INPUT  X. 

*•  ;  ER-OR  CHECK  .  IF  c  *  0.  U.K.,  IF  E»l.,  THE 

MATRIX  cr  CGiFFIClENTS  ARE  SINGULAR.  IF  E  *  2 . 
SOLUTION  is  ATTEMPTED  BUT  EQUATIONS  HAY  BE 
SINGULAR  \VJ  ILL  CCNOITIUNCD. 

R  E  M«  J.  \  S 
REFE  ;  *.  NCc 

INTjRNAL  TECHNICAL  Ni.T C  Rr> 

USUi.LE  PRECISION  SIMULT  AhEliLS  EQUATION  SOLVER  UR  MATRIX 
INVERSION  FuUTlNr  No.  1.15  ACG  IV&3. 

G .  G.  GlrthS 

SCIENTIFIC  DIGITAL  PhD  C»R  AM  w  I  NG  BRANCH 
A R“Y  COMPUTATION  CcNTr? 

RcCiTcNE  Ah  SEN  AL  ,  ALABAMA  ^ 

C'!‘,Ii“T  ^-<^c'Aac Jx' 

CHUF.THEURtTICAL  PkuGLtMS  SECTION  9 

SCIENTIFIC  DIGITAL  p:.JGbA“MIN&  BRANCH 
AkVY  COMPUTATION  CENTER 
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*EGR 

SCSDSd  -  IfN  SOURCE  STATEMENT  -  ti-NISI 


C 

c 


21 

6 


14 


15 
4 

16 


35 

36 


26 


3uo< JUT INE  SESOMII X.N.N3.MS.MN1 »C ,H , E .WORK . i HLD I 
01  Mi  NS  UN  XtMNl.ll  .-OAKIll.lHLOlil 

IE  COJHLE  PRECl  St  (IN  IS  DES1REC  INSERT  THE  filLLUWlNG  CARO 
OuJiLE  PRECISION  XtMnRKty.0tSUM.31 
UuUiilE  PRECISION  X .WORK, V, 0 .SUM t XI 
Xi-i. 
t-Q. 

R-0« 

00  21  I-l.N 
IHLOI  1  )  -1 
If  IMS! 6*4,6 

NU-.N-N 

<.B-N 


KU-.N*! 

Cl.  14  1*1  ,N 
OU  14  J-MN.KN 
XC I.JI-V.DC 
Jj  14  I-l.N 
J-  l*N 

XI  l.JI-l.OO 
GJ  TO  16 
Wi*'i*N  u 
JJ»*.N 
I.NN-N-l 
0-1.00 
0.)  36  I-l.N 

y-xu,  ii 

01.  35  J-2.N 

lECAssm.LT.ABsixu.jmy-xu  ,ji 

continue 

c-u-y 


00  3o  J-l.NN 
XI  I. J I -XI  I . Jl/Y 
30  “  I-l.N 
KK-N-I 

IF (KK I  13. 1C. 26 
LL-**M 
IJJ-i 
l- I 


mJRj.-X 

33  17  II-l.LL 
00  17  J-l.Ll 

If  UoSUDRKI-ABSIXII  I,  Jill  18,17,17 
Id  -..RX-XIII.JI 
L«J>I“ 1' 

IJJ-J 

17  CONTINUE 

IFIIJJ-112,2, 19 
19  00  ZZ-  II-1.N 
Y-XI 11,11 
XI  U.II-XIII.IJJ) 

23  XI  II, IJJI-Y 
IV-IHLOI I  I 
IHLOI  I I-IHLOILI 
lHLOILI«iy 

n— o 


SCS00223 

SES0C23C 

SES0U26C 

SES0C27C 

SFSQC2RC 

SES00290 

SESO0300 

SFSPC310 

SFS00320 

SES00330 

SES0C34D 

SES00350 

SFS00360 

SES00370 

SES00380 

SES00390 

SES00400 

SES00420 

SES004J0 


SES00440 

SES00450 

SES03460 

SES0047C 

SES00480 

SES00493 

SES00500 

SES00510 

SES00520 

SESUC530 

SES00540 

SES00550 

SES00560 

SES00570 

SES00590 

SES00590 

SESC10600 

SESOOSIO 

SES03620 

SES00633 

SES00640 

SES00650 

SES00660 
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..•..SV.r  -  :k<  jt.OHC  f  STATE.4E.Nf  -  1 F  NC  S I  - 

2  ;  .  l  » i . »  4 

I.  ^Sl  0-A4SIML*  1 .  ll>»7,  1  ,  l 

7  : 

i  J  *  1  »  J  J 

t  *  «<  1 .  J  I 
A  I  «,  J  I  ■  *t  k»  1,  J> 
i  *u»l. j»«r 

:  r. -..tenu: 

4' 

i>  i&j  tu 

ji  1  *-(-•» s i  iast  1* i-x  i/x;  1-1 . ) .  1  r.i  .t-m*;>4 

< :  =  ■ 

44  <*'*44 
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